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I.  INTRODUCTION 


A.  BACKGROUND 

Over  the  past  several  decades,  the  United  States  has  transitioned  from  a  goods- 
based  economy  to  a  services-based  economy.  As  of  2005,  services  accounted  for  more 
than  78%  of  the  country’s  gross  domestic  product  and  employed  80%  of  the  country’s 
workforce  (Coalition  of  Service  Industries  [CSI],  2007).  Federal  spending  on  services 
has  also  sharply  increased.  Most  recently,  the  Department  of  Defense  (DoD)  reported 
obligating  $212  billion  on  service  contracts  during  2009,  an  amount  that  accounted  for 
more  than  50%  of  the  DoD’s  2009  contract  spend  (Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  [USD(AT&L)],  2010a,  2010b).  In  that  year 
alone,  the  DoD  utilized  more  than  100,000  contract  vehicles  to  acquire  services  from 
32,000  different  service  suppliers  (USD[AT&L],  2010b). 

Despite  these  substantial  increases  in  spend  on  contract  services,  the  size  of  the 
DoD’s  acquisition  workforce  has  decreased  in  recent  years.  As  seen  in  Figure  1,  the 
DoD’s  spend  on  services  acquisitions  in  real  dollars  grew  more  than  100%  between  2001 
and  2008,  increasing  from  $92  billion  to  slightly  more  than  $200  billion,  while  the  DoD’s 
contracting  workforce  grew  only  1%  and  the  DoD’s  total  acquisition  workforce  shrank 
nearly  3%  (Government  Accountability  Office  [GAO],  2009b).  The  Government 
Accountability  Office  (GAO)  contends  that  the  DoD  carried  out  this  downsizing  without 
regard  to  retention  of  the  specific  skills  and  competencies  needed  to  accomplish  the 
Department’s  mission  (GAO,  2007a),  such  as  the  skills  required  to  manage  increasingly 
complex  service  contracts. 

At  the  same  time  that  demand  for  services  has  rapidly  increased  within  the  DoD, 
so  have  the  demands  on  service  contract  administrators.  Some  of  this  burden  can  be 
attributed  to  the  uniqueness  of  services — the  properties  of  intangibility,  heterogeneity, 
perishability  of  output,  and  simultaneity  of  production  and  consumption — all 
characteristics  that  differentiate  services  from  the  acquisition  of  goods  (Ellram,  Tate,  & 
Billington,  2007).  Because  of  these  attributes,  it  is  more  difficult  to  control  quality 
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levels,  more  difficult  to  evaluate  quality,  more  difficult  to  manage  service  personnel,  and 
more  difficult  to  manage  time,  which  is  the  process  of  synchronizing  the  resources 
required  for  service  delivery  with  the  time  of  consumption  (Lovelock  &  Wirtz,  2007). 

Dollars  are  constant  fiscal  year  2008  dollars  (in  billions) 

Workforce  (in  thousands) 

400 


Fiscal  year 

|  Products 
|  Servtces 

— ■  Contracting  career  field 

GAO  analysis.  Federal  Procurement  Data  System-Next  Generation,  DOO 

Figure  1.  Changes  in  DoD’s  Contract  Obligations  and  Contracting  Workforce,  Fiscal 

Year  2001  to  Fiscal  Year  2008 

(GAO,  2009a) 

This  study  intends  to  determine  the  explanatory  structure  of  services  sourcing 
performance,  specifically  that  of  service  quality  and  contracting  compliance  within  the 
U.S.  Air  Force.  Although  significant  research  has  been  conducted  in  the  area  of  service 
quality,  the  relationships  between  contract  administration  functions  and  service  contract 
outcomes  have  yet  to  be  examined.  This  is  not  surprising  given  the  number  of  variables 
potentially  involved  and  the  complexity  of  their  interaction.  However,  the  need  for  this 
knowledge  has  become  increasingly  more  critical  in  recent  years. 
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PROBLEM  STATEMENT  AND  GAP  IN  LITERATURE 


With  service  contract  administration  challenges  mounting,  the  DoD  has  recently 
come  under  scrutiny  by  the  GAO  regarding  the  acquisition  and  management  of  service 
contracts.  In  2001,  the  GAO  labeled  the  DoD’s  acquisition  of  services  as  -high  risk” 
(GAO,  2001a),  stating  that  the  department’s  poor  management  of  service  contracts 
undermined  the  government’s  ability  to  obtain  value  for  the  taxpayer’s  dollar.  Additional 
findings  in  the  2001  report  highlighted  the  DoD’s  difficulties  in  defining  requirements  as 
well  as  in  providing  sufficient  contractor  oversight  and  adequately  staffing  contracting 
professionals. 

A  second  GAO  report  in  2001  reiterated  findings  concerning  DoD  and  federal 
deficiencies  in  service  contract  management  and  made  three  recommendations:  appoint  a 
chief  acquisition  officer  for  each  agency,  improve  training  of  the  acquisition  workforce, 
and  increase  the  use  of  performance-based  contracting  (GAO,  2001b).  As  defined  by  the 
Office  of  Federal  Procurement  and  Public  Policy  (OFPP),  perfonnance-based  contracts 
describe  contractual  requirements  in  terms  of  the  government’s  desired  results  and 
measurable  outcomes,  and  they  establish  procedures  to  manage  perfonnance  if  it  falls 
below  established  thresholds.  If  appropriate,  performance-based  contracts  can  use 
incentives  to  motivate  contractor  efforts  that  otherwise  might  not  be  emphasized  and  to 
discourage  waste  and  inefficiency.  At  the  time  of  the  GAO  report,  in  2001,  a  mere  15% 
of  federal  service  contracts  were  using  performance-based  contracting  methods,  despite 
the  Office  of  Management  and  Budget’s  (OMB)  20%  goal  (GAO,  2001b).  To  date, 
improvements  in  acquisition  training  and  in  the  implementation  of  the  role  of  the  chief 
acquisition  officer  have  not  been  fully  realized  (see,  for  example,  GAO,  2010b;  Falcone, 
2010). 

The  following  year,  in  2002,  the  GAO  asked  five  of  the  major  government 
agencies,  including  the  DoD,  to  identify  contracts  that  exemplified  performance-based 
service  contracting  (GAO,  2002b).  The  GAO  reviewed  25  of  these  contracts  and  found 
that  only  9  of  25  exhibited  all  of  the  attributes  of  performance-based  service  contracts  as 
defined  by  the  OFPP.  Of  the  remaining  16  contracts,  4  did  not  exhibit  any  of  the 
attributes.  It  was  evident,  based  on  this  study,  that  agencies  were  only  partially 
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complying  with  the  OMB’ s  guidance  to  implement  the  use  of  performance-based 
contracting  procedures. 

Also  in  2002,  the  GAO  released  a  study  conducted  on  six  leading  companies  that 
had  instituted  a  strategic  approach  to  the  acquisition  of  services.  Brunswick  Corporation, 
Dun  &  Bradstreet  Corporation,  Electronic  Data  Systems  Corporation,  Exxon  Mobile 
Corporation,  Hasbro  Incorporated,  and  Merrill  Lynch  &  Company  Incorporated  have  all 
successfully  reengineered  their  business  practices  for  acquiring  services  and,  as  a  result, 
have  netted  cost  savings  as  high  as  15%  while  maintaining  or  even  improving  service 
levels  (GAO,  2002a).  The  GAO  found  similar  key  elements  among  the  strategic 
approaches  adopted  by  these  companies;  consequently,  it  recommended  that  the  DoD 
incorporate  these  elements  as  a  general  framework  for  services  acquisition  reform. 

As  seen  in  Figure  2,  commitment  to  taking  a  strategic  approach  to  services 
acquisition  was  central  to  the  successful  implementation  of  change  within  the  six 
companies  studied.  The  companies  first  had  to  secure  buy-in  from  top  corporate 
leadership,  who  could  communicate  the  urgency  for  change  and  establish  acquisition 
goals  and  targets  for  the  company.  The  framework  followed  a  logical  order  for  process 
implementation: 

•  Develop  the  strategic  structure,  processes,  and  roles;  establish  business 
relationships  between  all  services  stakeholders;  identify  service  experts. 

•  Support  the  people  and  processes  involved;  maintain  open  communication 
between  parties;  use  metrics  to  reinforce  process  credibility. 

•  Employ  information  systems  to  uncover  the  who,  what,  and  where  of 
spending;  analyze  spend  data  to  identify  opportunities  for  increased  value. 


In  this  same  study,  the  GAO  compared  the  DoD’s  current  service  acquisition 
environment  to  that  of  the  six  companies  prior  to  their  implementation  of  a  strategic 
approach  to  services  acquisition.  The  report  contended  that  the  enterprise-wide,  one-size- 
fits-all  solution  adopted  by  these  companies  may  not  be  suitable  for  the  DoD,  but  stressed 
that  the  Department  develop  a  framework  for  services  reform  using  a  strategic  approach 
modeled  after  successes  in  industry. 
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However,  more  than  four  years  later,  in  November  2006,  the  GAO  issued  another 
report  stating  that  the  DoD  still  had  not  implemented  a  strategic  approach  to  the 
management  of  service  contracts  (GAO,  2006b).  Rather,  the  GAO  found  that  the  DoD 
was  reactively  managing  its  service  contracts,  due  in  part  to  a  lack  of  information  on 
service  requirements,  volume,  and  composition.  The  DoD  concurred  with  these  findings 
and  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD[AT&L])  issued  a  policy  memorandum  to  strengthen  the  management  of  DoD 
services  acquisition  (USD[AT&L],  2006).  This  memorandum  called  for  reform  at  the 
strategic  and  tactical  levels  to  ensure  services  acquisitions  were  enhancing  the 
capabilities  of  the  warfighter  and  to  achieve  specific  objectives  such  as  the  use  of 
perfonnance-based  measures,  contract  action  reporting,  and  regulatory  compliance. 
Additionally,  the  memorandum  implemented  the  changes  required  by  Section  812  of  the 
National  Defense  Authorization  Act  for  Fiscal  Year  2006,  which  required  the  DoD  to 
establish  a  management  structure  for  the  acquisition  of  contract  services. 


Figure  2.  Key  Elements  of  Strategic  Approach  Taken  by  Leading  Companies 

(GAO,  2002a) 
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Despite  these  changes,  the  GAO  still  saw  little  improvement  in  the  DoD’s 
acquisition  and  management  of  services.  In  2007,  the  GAO  reported  that  the  DoD 
increasingly  relied  on  defense  contractors  for  business-to-business  services  but  lacked  the 
key  elements  at  the  strategic  and  tactical  levels  to  make  service  contracts  a  managed 
outcome  (GAO,  2007a).  These  results  were  reiterated  in  another  2007  GAO  report 
(GAO,  2007b),  which  questioned  whether  the  DoD  applies  sound  business  practices  to 
the  acquisition  and  management  of  contracted  services  in  the  following  competencies: 

•  Defining  requirements, 

•  Obtaining  adequate  competition, 

•  Managing  contractors  in  a  deployed  (contingency)  environment, 

•  Assessing  contractor  performance,  and 

•  Executing  interagency  contracts  and  task  orders. 

Recent  legislation  passed  within  the  National  Defense  Authorization  Act  of  2008 
required  the  DoD  to  submit  to  Congress  an  annual  inventory  of  contracted  services  and 
also  required  the  Secretary  of  Defense  to  issue  guidance,  with  detailed  implementation 
instructions,  providing  for  periodic  independent  management  reviews  of  the 
Department’s  service  contracts.  The  DoD’s  resultant  guidance  memorandum  required 
contract  peer  reviews  at  the  DoD  level  for  service  acquisitions  above  $1  billion  and 
placed  the  responsibility  of  conducting  reviews  within  each  component  for  all  other 
service  acquisitions  falling  below  that  threshold.  However,  the  GAO  recently  contested 
the  effectiveness  of  these  policies,  asserting  that  the  same  pattern  of  service  acquisition 
mismanagement  still  exists  within  the  DoD  (GAO,  2009a).  In  April  2009,  the  GAO 
published  a  testimony  before  Congress  on  the  DoD’s  progress  toward  improving  its 
acquisition  of  contract  services.  In  this  report,  the  GAO  contended  that  the  DoD  has 
made  little  headway  on  its  longstanding  issues  in  service  contracting  (GAO,  2009a).  This 
report  cited  many  of  the  same  issues  as  previous  reports  and  focused  on  the  Department’s 
challenges  employing  sound  business  practices,  such  as  defining  requirements,  selecting 
contract  type,  and  adequately  overseeing  contractor  performance. 
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The  DoD  has  also  been  highly  scrutinized  for  its  lack  of  compliance  with 
contracting-related  statutes  and  regulations.  Recent  reports  have  highlighted  compliance 
issues  ranging  from  a  lack  of  required  documentation,  such  as  determinations  and 
findings  for  the  use  of  time  and  materials  contracts  (GAO,  2007c),  to  issues  meeting 
competition  requirements  (Department  of  Defense  Inspector  General  [DoDIG],  2004, 
2010a;  GAO,  2004)  and  managing  and  definitizing  undefinitized  contractual  actions 
(DoDIG,  2010b;  GAO,  2007d,  2010a).  Many  of  the  other  reports  related  to  services 
acquisition  have  focused  on  the  application  of  performance-based  services  acquisition 
(PBSA)  procedures,  such  as  a  recent  GAO  report  (2009b)  that  identified  challenges  faced 
by  the  DoD  in  applying  PBSA  procedures  to  professional  and  management  support 
contracts  and  task  orders. 

But  why  is  the  DoD  experiencing  so  much  trouble  with  the  acquisition  and 
management  of  contract  services?  Recent  literature  has  uncovered  similar  problems  for 
firms  purchasing  business-to-business  (B2B)  services  in  the  commercial  marketplace. 
Like  the  DoD,  firms  are  spending  increasingly  more  on  outsourced  services,  but  the 
resources  dedicated  to  managing  those  services  have  not  kept  pace  (Ellram  et  ah,  2007). 
Additionally,  overworked  services  acquisition  personnel  often  use  supplier-provided 
information  to  determine  the  cost  structure  and  cost  drivers  of  acquired  services  (Ellram 
et  al.,  2007).  An  opportunistic  service  firm  can  take  advantage  of  this  circumstance  to 
reduce  the  resources  applied  to  perform  the  service  at  the  expense  of  the  purchasing  firm. 

To  address  these  issues,  Ellram  et  al.  (2007)  called  for  firms  that  purchase  B2B 
services  to  take  six  steps  to  improve  services  supply  management.  First,  firms  should 
conduct  a  spend  analysis  to  determine  on  what  services  company  resources  are  being 
spent,  who  the  customers  of  the  service  are  within  the  firm,  the  level  at  which  services 
acquisitions  are  approved  within  the  organization,  the  structure  of  services  management 
personnel,  and  their  relationships  to  service  performance.  The  second  step  is  for  firms  to 
segment  their  services  spending  based  on  the  value  to  the  organization  and  on  the  level  of 
risk,  in  part  to  determine  the  appropriate  level  of  interaction  between  the  users 
(requesters)  of  the  service  and  the  service  supply  personnel.  Similarly,  in  the  third  step, 
firms  should  allocate  resources  toward  the  management  of  the  service  requirements  in  a 
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manner  consistent  with  economic  return.  For  example,  in  developing  requirements 
documents,  a  firm  should  use  more  of  its  own  resources  on  high-risk,  high-importance 
services  than  on  those  that  provide  very  little  value  or  present  little  risk  to  the 
organization.  Fourth,  Ellram  et  al.  (2007)  called  for  firms  to  increase  the  professionalism 
of  their  services  management  personnel.  And  in  the  fifth  step,  they  advised  that  firms 
should  measure  effectiveness  and  ensure  proper  business  controls  through  the  use  of 
supplier  audits,  information  technology,  and  the  incorporation  of  concrete  performance 
requirements  into  contracts.  Finally,  Ellram  et  al.  (2007)  argued  that,  as  a  sixth  step, 
firms  should  recruit  and  place  their  best  talent  into  services  supply  management. 

However,  a  general  lack  of  knowledge  by  public  and  private  purchasers  in  the 
subject  matter  incorporated  into  several  of  these  steps  results  in  unique  challenges.  For 
example,  purchasing  organizations  know  neither  how  many  resources  are  appropriate  to 
allocate  toward  the  acquisition  and  management  of  services  nor  how  to  measure  the 
effectiveness  of  services.  Furthermore,  in  order  to  improve  the  performance  of  service 
contract  management  within  the  U.S.  Air  Force,  or  within  the  DoD  as  a  whole,  the 
determinants  of  performance  need  to  be  identified.  Once  these  are  known,  leaders  in 
public  procurement  can  more  effectively  allocate  resources  toward  those  factors  that  have 
the  greatest  impact  on  performance  and  avoid  the  inefficient  use  of  resources  on  those 
factors  that  have  little  or  no  impact. 

C.  RESEARCH  OBJECTIVES 

The  objective  of  this  research  is  to  address  existing  gaps  in  the  literature  and  to 
offer  service  contract  practitioners  within  the  U.S.  Air  Force  a  comprehensive  model  to 
use  in  improving  the  acquisition  of  services  and  increasing  compliance  with  federal, 
DoD,  and  Service  acquisition  regulations.  This  study  will  answer  the  following  research 
questions: 

Research  Question  1:  What  are  the  determinants  of  sendees  sourcing 

performance? 
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Research  Question  2:  Which  determinant(s)  have  the  greatest  impact  on  the  key 
contract  outcomes  of  service  quality  and  regulatory  and  statutory  compliance? 

Research  Question  3:  How  can  service  quality  be  validly  measured  in  a  business- 
to-business  context? 

D.  METHODOLOGY 

This  research  provides  a  quantitative  examination  of  the  determinants  of  service 
sourcing  performance  using  structural  equation  modeling  (SEM).  SEM  is  a  multivariate 
statistical  method  that  extends  confirmatory  factor  analysis  to  test  causal  relationships 
between  combinations  of  manifest  and  latent  variables.  Latent  variables  are  those 
variables  that  are  not  directly  observable,  but  that  are  inferred  from  a  set  of  manifest 
(observed)  variables.  They  are  often  measured  through  the  application  of  a  pre-defined 
scale  when  a  survey  is  utilized  for  the  data  collection.  In  general,  both  latent  and 
observed  variables  that  are  unexplained  in  a  model  are  considered  to  be  exogenous. 
Exogenous  variables  have  one  or  more  causal  paths  that  lead  from  them  to  endogenous 
variables.  Endogenous  variables  are  explained  in  the  model  and  have  one  or  more 
hypothesized  causal  paths  leading  to  them  from  other  endogenous  or  exogenous 
variables.  In  path  analysis  and  in  SEM,  an  endogenous  variable  may  be  both  explanatory 
and  explained  within  a  model,  when  such  a  variable  is  dependent  on  certain  endogenous 
or  exogenous  variables  while  explaining  one  or  more  other  variables.  Indeed,  the  ability 
to  model  a  series  of  dependent  variables  is  a  unique  strength  of  SEM.  In  this  study, 
endogenous  variables  consisted  of  the  service  acquisition  outcomes  of  service  quality  and 
compliance  with  regulations  and  statutes  as  well  as  with  the  sufficiency  of  the 
requirement  definition.  Several  exogenous  latent  and  manifest  variables  were  also  used 
in  the  study  and  stemmed  from  theories  presented  in  the  literature  review. 

Existing  literature  was  used  to  develop  hypotheses,  a  conceptual  model,  and  a 
survey,  all  of  which  were  later  pre-tested  and  refined  through  interviews  with  practicing 
contract  administrators.  The  final  survey  was  deployed  online  to  a  population  of  U.S.  Air 
Force  contract  administrators.  Usable  responses  were  collected  from  240  participants, 
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and  survey  data  were  initially  analyzed  using  exploratory  and  confirmatory  factor 
analysis.  Finally,  the  structural  model  was  tested  for  overall  model  fit  as  well  as  for  the 
significance  of  individual  causal  paths;  a  trimmed  and  final  structural  model  is  presented 
in  this  thesis  for  parsimony  and  future  replication. 

E.  ORGANIZATION  OF  THE  STUDY 

A  literature  review  is  first  presented  in  Chapter  II.  Recent  literature  concerning 
service  quality  is  discussed  and  brief  overviews  are  provided  on  several  applicable 
theories,  including  agency  theory,  the  resource-  and  competence-based  views  of  the  firm, 
and  relational  exchange.  The  research  methodology,  including  development  of  a 
conceptual  model,  sampling,  and  survey  design,  is  located  in  Chapter  III;  the  results  of 
the  data  collection  and  analysis  are  located  in  Chapter  IV.  Finally,  managerial  and 
theoretical  implications  and  several  recommendations  for  future  research  are  discussed  in 
the  concluding  chapter,  Chapter  V. 

F.  SCOPE 

Literature  addressing  service  quality  has  generally  been  limited  to  examinations 
of  practices  within  the  setting  of  commercial  industry.  However,  due  to  limits  on  time 
and  resources,  the  scope  of  this  research  was  restricted  to  the  study  of  services  sourcing 
perfonnance  within  the  U.S.  Air  Force.  Furthermore,  this  study  is  focused  on  services 
contracting  within  the  United  States  and  excludes  overseas  contracting  activities  because 
cultural  and  business  norms,  as  well  as  standards  for  the  application  of  contracting 
procedures,  may  differ  in  other  countries.  The  services  acquisitions  analyzed  in  this 
study  were  conducted  using  the  relevant  mandatory  rules  and  procedures  for  contracting 
within  the  DoD,  including  the  procedures  established  in  the  Federal  Acquisition 
Regulation  (FAR),  the  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS), 
and  any  applicable  U.S.  Air  Force  supplements.  Such  a  study  in  a  government  context  is 
appropriate  because  the  standardized  rules  and  procedures  can  better  exclude  the  type  of 
systematic  bias  that  is  commonly  found  due  to  widely  differing  contract  management 
processes  across  commercial  firms. 
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G.  MANAGERIAL  AND  THEORETICAL  IMPLICATIONS 

A  study  that  identities  the  determinants  of  key  service  contract  outcomes  can  offer 
tremendous  utility  to  practitioners.  First,  the  development  of  a  model  for  services 
sourcing  performance  may  assist  in  the  effective  assignment  of  contract  procurement  and 
administration  resources  to  service  contracts.  It  may  also  provide  a  framework  that  can 
be  used  to  assess  the  effectiveness  of  regulations  and  statutes  on  the  quality  of  contracted 
services.  Some  of  the  findings  (e.g.,  a  strong  relationship  between  the  commitment  of  the 
internal  customer  and  the  sufficiency  of  the  requirement  definition)  suggest  that  a 
paradigm  shift  in  the  assignment  of  upstream  resources — such  as  those  assigned  by 
requiring  activities — may  be  fundamental  to  improving  service  contract  outcomes. 
Further  findings  suggest  that  adequate  communication  between  the  contractor  and 
government  acquisition  personnel  is  fundamental  to  achieving  the  government’s  desired 
level  of  service  performance  and  to  achieving  full  compliance  with  acquisition  statutes 
and  regulations.  The  amount  of  turnover  of  acquisition  personnel  was  also  found  to  be 
directly  related  to  the  reported  level  of  compliance;  acquisitions  with  more  than  100% 
turnover  across  the  life  of  the  contract  or  with  more  than  42%  turnover  annually  were 
found  to  be  significantly  less  compliant  with  statutes  and  regulations.  Furthermore,  low 
buyer  experience  was  found  to  be  significantly  related  with  lower  levels  of  service 
quality,  as  was  the  amount  of  compliance  with  regulations  and  statutes. 

In  addition  to  the  managerial  implications,  this  study  addresses  a  theoretical  need 
for  additional  research  of  service  quality  in  B2B  contexts — a  field  where  previous 
research  on  the  determinants  of  service  quality  has  been  sparse.  A  new  framework  is 
proposed  in  this  thesis  for  the  sourcing  of  B2B  services — a  framework  that  is  based  on 
the  works  of  Kong  and  Mayo  (1993)  and  Parasuraman,  Zeithaml,  and  Berry  (1985).  This 
framework  presents  the  gaps  that  exist  between  buyers  and  suppliers  in  B2B  service 
acquisitions  when  service  requirements  are  defined  by  the  purchasing  organization. 
Additionally,  a  more  relevant  measure  of  service  quality  was  needed  for  studies  of  B2B 
services;  many  previous  researchers  have  simply  excluded  measurement  dimensions  from 
an  existing  B2C  service  quality  scale  without  giving  consideration  to  the  unique  aspects 
of  B2B  service  acquisitions.  As  such,  a  new  service  quality  scale  is  proposed  in  this 
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study  that  incorporates  the  measurement  of  the  service  supplier’s  responsiveness  to  the 
buyer’s  requirements,  a  dimension  that  is  particularly  applicable  to  measuring  quality  in 
the  acquisition  of  services  that  are  not  available  off-the-shelf  and  when  perfonnance 
requirements  are  defined  by  the  purchasing  organization. 

H.  CONCLUSION 

In  summary,  this  study  uses  structural  equation  modeling  to  identify  several 
determinants  of  service  quality  and  regulatory  and  statutory  compliance  through  a 
quantitative  analysis  of  survey  data.  With  the  literature  gap  and  problem  statements 
identified,  the  following  chapter  discusses  the  theories  that  are  relevant  to  this  study  and 
introduces  the  research  hypotheses. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

Multiple  theories  are  often  relied  upon  to  explain  B2B  exchange.  The  following 
theories  are  relevant  to  services  procurement:  service  quality  theory,  relational  exchange 
theory,  agency  theory,  the  resource-based  view  of  the  firm,  and  the  competence-based 
view  of  the  firm.  Hypotheses  for  this  study  are  proposed  in  each  section  of  this  chapter 
and  are  later  summarized  in  the  conclusion. 

B.  NATURE  OF  SERVICES 

Quality  is  an  immensely  important  yet  fleeting  measure  for  firm  success 
(Anderson  &  Zeithaml,  1984)  in  that  it  is  not  easily  articulated  by  service  providers  or  by 
customers  (Takeuchi  &  Quelch,  1983).  This  is  especially  true  when  it  comes  to  the 
quality  of  services  because  they  differ  drastically  in  nature  from  goods,  primarily  through 
the  four  characteristics  of  intangibility,  heterogeneity,  perishability,  and  inseparability 
(Ellram  et  ah,  2007). 

First,  the  intangible  nature  of  services  renders  specifications  and  customer 
expectations  to  be  imprecise  (Ellram  et  ah,  2007).  A  purchaser  of  office  furniture  may 
have  little  trouble  articulating  customer  requirements  for  a  desk  and  may  be  able  to 
employ  several  senses  to  determine  the  quality  of  a  good.  However,  it  may  be  impossible 
for  an  individual  requiring  an  appendectomy  to  establish  a  desired  level  of  performance 
for  the  surgical  procedure.  This  is  due  not  only  to  a  knowledge  gap  between  the  provider 
and  receiver  of  the  service  but  also  to  the  little  physical  evidence  that  exists  of  the  service 
provided. 

Second,  services  are,  by  nature,  heterogeneous.  This  is  especially  true  of  services 
with  a  high  labor  content  (Parasuraman  et  ah,  1985)  because  performance  will  vary 
between  providers  and  will  likely  differ  between  customers  and  with  time.  Like 
providers,  customers  lack  a  homogenous  definition  of  service  quality  for  any  specified 
service.  Because  of  this,  and  because  consistency  in  levels  of  performance  from  service 
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personnel  is  difficult  to  attain,  the  level  of  quality  that  a  service  provider  expects  to 
deliver  may  vary  greatly  from  the  level  of  quality  that  the  customer  expects  to  receive 
(Parasuraman  et  ah,  1985). 

Third,  services  are  frequently  perishable;  unlike  goods,  services  cannot  be  held  or 
stocked  in  inventory.  While  inventory  policies  for  goods  allow  firms  to  buffer  future 
demand  with  safety  stock,  service  providers  must  change  service  capacity  to  meet 
demand  fluctuations  (Ellram  et  ah,  2007).  The  perishability  of  services  also  presents 
challenges  for  inspection;  service  delivery  or  outcomes  for  many  services  can  only  be 
inspected  or  evaluated  at  the  time  the  service  is  performed.  For  example,  security  guard 
services  cannot  be  rendered  for  a  previous  period  of  time  nor  can  the  services  be  easily 
inspected  or  evaluated  after  the  period  of  performance  is  complete. 

Fourth,  the  production  and  consumption  of  services  are  often  inseparable.  During 
the  production  of  a  good,  such  as  a  television  set,  quality  can  be  engineered  into  the  end 
product  at  a  factory  and  then  delivered  intact  to  the  customer.  The  customer’s  input  into 
the  quality  of  the  television  will  have  little  or  no  impact  on  the  good  produced. 
Alternatively,  quality  in  a  service  environment  often  occurs  through  interactions  between 
the  customer  and  the  service  provider  (Lehtinen  &  Lehtinen,  1982).  These  interactions 
become  even  more  influential  on  quality  for  services  in  which  higher  degrees  of  customer 
input  are  required  (Parasuraman  et  ah,  1985),  such  as  with  real  estate  or  cosmetology 
services. 

C.  SERVICE  QUALITY 

Parasuraman  et  al.  (1985)  performed  an  examination  of  literature  surrounding 
service  quality  and  suggested  the  following  three  underlying  themes  to  defining  service 
quality: 

•  -Service  quality  is  more  difficult  for  the  consumer  to  evaluate  than  goods 
quality.”  (p.  42) 

•  -Service  quality  perceptions  result  from  a  comparison  of  consumer 
expectations  with  actual  service  performance.”  (p.  42) 
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•  -Quality  evaluations  are  not  made  solely  on  the  outcome  of  a  service;  they 
also  involve  evaluations  of  the  process  of  service  delivery.”  (p.  42) 

Parasuraman  et  al.’s  (1985)  analysis  and  measurement  of  these  themes  included 
the  notion  of  expectation-disconfinnation  theory,  or  the  gap  between  customer 
expectations  and  perceptions  of  actual  performance.  Other  researchers  have  also 
considered  that  the  delivery  of  quality  services  means  minimizing  the  expectation- 
disconfirmation  gap  on  a  consistent  basis  (Lewis  &  Booms,  1983)  and  that  multiple 
forms  of  service  quality  may  exist.  For  example,  technical  quality,  which  is  what  the 
customer  actually  receives  from  the  performance  of  the  service,  may  differ  from  the 
method  of  service  performance,  defined  as  functional  quality  (Gronroos,  1982). 
Alternatively,  Lehtinen  and  Lehtinen  (1982)  offer  a  three-dimensional  view  of  service 
quality  consisting  of  physical  quality,  interactive  quality,  and  corporate  quality,  the  last  of 
which  they  define  as  the  customers’  perceptions  of  the  service  firm’s  image. 

Parasuraman  et  al.  (1985)  developed  a  conceptual  model  of  service  quality 
(Figure  3),  hereafter  referred  to  as  the  Gaps  Model,  which  was  developed  from  an 
exploratory  investigation  and  is  based  on  the  gaps  between  corporate  executives’ 
perceptions  of  service  tasks  and  quality  and  their  expectations  of  the  same.  The  first  gap 
presented  in  this  model  is  a  gap  between  the  consumer’s  expectations  of  the  service  and 
the  service  provider’s  perception  of  the  consumer’s  expectations.  Parasuraman  et  al. 
(1985)  found  that  service  provider’s  perceptions  of  consumer  expectations  were  often 
accurate;  however,  discrepancies  exist.  The  second  gap  is  the  difference  between  the 
service  provider’s  perceptions  of  consumer  expectation  and  the  translation  of  those 
perceptions  into  corporate  specifications  for  service  quality.  This  gap  can  often  be 
attributed  to  the  uncertainty  of  demand  for  a  service,  but  Parasuraman  et  al.  (1985)  also 
theorized  that  it  is  due  to  a  lack  of  total  commitment  among  management  to  deliver 
quality  services.  The  third  gap  is  the  difference  between  the  corporate  specifications  for 
service  quality  and  the  actual  service  perfonnance.  In  Parasuraman  et  al.’s  (1985) 
investigation,  executives  of  service-delivery  firms  routinely  stressed  the  critical  function 
of  service  employees.  Therefore,  the  depth  of  this  gap  is  contingent  on  the  difference 
between  the  performance  of  the  firm’s  service  delivery  employees  and  the  specifications 
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established  by  the  firm.  The  authors’  fourth  gap  is  the  difference  between  advertized 
(i.e.,  promised)  service  delivery  and  actual  service  performance.  This  gap,  relating  the 
use  of  media  to  the  expectancy-disconfirmation  paradigm,  suggests  that  a  consumer  will 
have  a  lower  perception  of  service  quality  if  the  firm  advertizes  an  inaccurately  high  level 
of  service  quality  through  external  communications.  Finally,  Parasuraman  et  al.  (1985) 
concluded  that  the  fifth  gap,  which  is  the  total  difference  between  the  consumer’s 
expected  level  of  service  and  the  perceived  level  of  service  received,  is,  by  design,  a 
function  of  all  earlier  gaps. 


CONSUMER 


Figure  3.  Gaps  Model 

(Parasuraman  etal.,  1985) 
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The  exploratory  research  of  Parasuraman  et  al.  (1985)  also  included  an 
investigation  into  the  consumer’s  view  of  service  quality,  in  which  they  uncovered  the 
following  10  common  detenninates: 

1 .  Reliability — consistency  of  performance,  dependability; 

2.  Responsiveness — timeliness  of  service  delivery,  readiness  of  employees; 

3.  Competence — possession  of  requisite  knowledge  and  skills; 

4.  Access — approachability  of  contact  personnel,  ease  of  contact; 

5.  Courtesy — politeness,  respect,  friendliness; 

6.  Communication — extent  that  the  customer  is  informed; 

7.  Credibility — reputation  of  firm  and  employees,  honesty,  trustworthiness; 

8.  Security — freedom  from  danger,  risk,  or  doubt; 

9.  Understanding/knowing  the  customer — comprehending  customer  needs; 
and 

10.  Tangibles — physical  evidence  of  service  perfonnance. 


Later,  Parasuraman  et  al.  (1988)  categorized  the  10  determinates  into  five 
dimensions:  tangibles,  reliability,  responsiveness,  assurance,  and  empathy.  Factors 
shaping  the  customer’s  expectations  of  service  quality  may  include  the  customer’s  past 
experience,  personal  needs,  and  word-of-mouth  communications.  They  developed  a  44- 
item  scale  for  the  measurement  of  service  quality  in  which  22  items  measure  customer 
expectations  of  service  quality  and  the  same  22  items  are  repeated  to  measure  customer 
perceptions  of  quality  from  actual  service  delivery.  It  is  important  to  note  a  paradigmatic 
limitation  of  the  Gaps  Model.  Going  back  to  the  first  gap,  the  presumption  is  that  the 
service  provider  must  anticipate  the  consumer’s  expectations.  Hence,  there  is  some 
uncertainty.  Yet,  this  presumption  does  not  apply  to  many  services.  For  example, 
customized  services  in  which  customers  and  service  providers  communicate  expectations 
and  capabilities  prior  to  exchange  (e.g.,  swimming  pool  installation),  deviate  from  this 
paradigm. 

In  contrast  to  the  Gaps  Model,  Cronin  and  Taylor  (1992)  argued  that  the 
expectancy-disconfirmation  theory  is  actually  a  measure  of  satisfaction  and  not  a  measure 
of  service  quality.  They  concluded,  using  an  empirical  study,  that  service  quality  directly 
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affects  customer  satisfaction,  which,  in-tum,  affects  the  customer’s  future  purchase 
intentions.  From  this  relationship,  they  theorized  that  perceived  service  quality  is,  in  fact, 
a  long-term  evaluation,  whereas  the  gap-based  model  measures  customer  satisfaction,  a 
short-term,  transaction-based  measure.  They  advocated  for  an  alternative  performance- 
based  paradigm  that  measures  service  quality  using  a  performance-only,  unidimensional 
score  rather  than  the  gap-based,  multidimensional  model  introduced  by  Parasuraman  et 
al.  (1985).  Cronin  and  Taylor’s  (1992)  position  is  based  on  the  notion  that  the  level  of 
service  quality  should  meet  customer  needs  first  and  foremost,  rather  than  simply  meet 
customer  expectations. 

While  the  works  of  Parasuraman  et  al.  (1985,  1988)  and  Cronin  and  Taylor  (1992) 
have  significantly  advanced  service  quality  theory,  their  research  has  been  limited  to 
applications  of  service  quality  in  business-to-consumer  (B2C)  contexts.  Kong  and  Mayo 
(1993)  recognized  a  need  for  a  deeper  theoretical  foundation  of  service  quality  in  B2B 
contexts  and  attempted  to  address  this  need  through  the  development  of  an  alternative 
framework,  hereafter  referred  to  as  the  B2B  Gaps  Model.  This  framework  was 
developed  as  an  extension  to  Parasuraman  et  al.’s  (1985)  Gaps  Model  and  involves 
account  services  rendered  between  channel  members  (e.g.,  manufacturers  and  retailers)  in 
a  supply  chain  for  physical  goods.  The  first  gap  in  the  B2B  Gaps  Model  (Figure  4)  is  the 
difference  between  the  management’s  perceptions  of  consumer  expectations  and  the 
actual  expectations  of  the  consumer  and  channel  member.  The  second  and  third  gaps  are 
retained  directly  from  the  Gaps  Model;  the  second  gap  is  the  difference  between 
manager’s  perceptions  of  consumer  expectations  and  the  translation  of  these  perceptions 
into  service  quality  specifications,  and  the  third  gap  is  the  difference  between  the 
translated  quality  specifications  and  the  actual  service  delivery.  The  fourth  gap  in  the 
B2B  Gaps  Model  is  an  extension  of  the  fourth  gap  from  the  Gaps  Model,  which  is  the 
difference  between  the  service  marketer’s  external  communications  to  consumers 
regarding  the  service  and  the  actual  service  delivery.  The  B2B  Gaps  Model  extends  this 
gap  by  acknowledging  that  supplier  communications  to  consumers  likely  differ  from 
those  to  channel  members.  The  final  gap  is  simply  the  difference  between  service 
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Figure  4.  B2B  Gaps  Model 

(Kong  &  Mayo,  1993) 


While  the  B2B  Gaps  Model  developed  by  Kong  and  Mayo  (1993)  addressed  the 
importance  of  the  channel  member  (e.g.,  a  retailer)  in  B2B  supply  chains,  its  frame  of 
reference  was  services  rendered  between  channel  members  (e.g.,  a  manufacturer  and  a 
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retailer)  rather  than  satisfaction  of  customer  demand.  In  practice,  many  organizations 
acquire  services  in  which  service  requirements  and  acceptable  levels  of  performance  are 
defined  by  the  customer,  as  is  standard  practice  in  public  procurement.  In  these 
instances,  the  channel  member  role  depicted  in  the  B2B  Gaps  Model  is  often  performed 
by  an  acquisition  professional  in  the  buying  organization  such  as  a  program  manager, 
who  oversees  the  procurement  from  the  early  planning  phases  to  the  evaluation  of 
suppliers,  selection  and  award,  payment  and  performance,  and,  finally,  to  contract 
closeout.  Because  this  role  is  not  considered  in  either  Kong  and  Mayo’s  (1993)  or 
Parasuraman  et  al.’s  (1985)  framework,  the  supply  chain  process  for  the  acquisition  of 
customer-defined  B2B  is  absent  of  supporting  theory. 

In  customer-defined  B2B  service  acquisitions,  organizational  requirements  are 
first  crafted  into  requirements  documents  based  on  an  agent’s  perception  of  the 
organization’s  needs.  A  gap  exists  at  the  point  in  which  there  is  potential  for  incongruity 
between  the  agent’s  perception  of  the  organization’s  requirements  and  the  organization’s 
actual  requirements.  A  second  gap  may  occur  between  the  agent’s  perception  of  the 
organization’s  requirements  and  the  sufficiency  with  which  those  requirements  are 
translated  into  service  specifications,  such  as  the  description  of  the  service(s)  to  be 
purchased  and  the  expected  level  of  quality. 

The  use  of  customer-defined  service  specifications  also  complicates  the  role  of 
the  supplier  in  the  B2B  framework.  First,  the  customer’s  requested  service  may  differ 
from  the  supplier’s  typical  methods  of  service  delivery  and  the  standards  of  quality  that 
the  supplier  customarily  performs.  In  some  cases,  a  knowledge  or  capability  disparity 
may  exist  between  the  supplier’s  customary  service  offerings  and  the  service 
requirements  of  the  customers.  In  other  cases,  the  customer  may  request  service  levels 
that  are  substantially  less  than  the  service  levels  that  the  supplier  typically  offers.  As 
such,  a  gap  exists  between  the  supplier’s  customary  service  and  the  resourcing  of  the 
supplier’s  management  or  work  breakdown  structure  (WBS).  The  actual  interpretation  of 
the  management’s  vector  by  the  supplier’s  employees  may  present  an  additional  gap,  in 
which  service  delivery  differs  from  the  WBS  or  the  management’s  resourcing. 
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Finally,  in  a  customer-defined  B2B  framework,  the  role  of  the  acquisition 
professional  is  central  as  a  liaison  between  the  customer  and  the  supplier.  This 
individual,  or  group  of  individuals,  has  expectations  of  service  delivery  that  may  differ 
from  the  end  customer’s  service  expectations.  For  one,  the  acquisition  professional  may 
rely  on  outside  sources  of  information  to  form  expectations,  such  as  research  of  the 
marketplace,  communications  with  the  supplier’s  employees  before  and  after  service 
perfonnance,  and  reviews  of  the  WBS  and  other  tools  provided  by  the  supplier’s 
management.  Internal  sources  also  shape  the  acquisition  professional’s  expectations, 
such  as  the  requirements  documents  and  performance  objectives.  The  difference  between 
the  level  of  service  that  this  acquisition  professional  expects  and  the  level  of  service  that 
is  actually  performed  is  the  final  gap  in  the  framework. 

As  such,  a  new  gaps  model  is  proposed  for  B2B  services  in  which  service 
requirements  are  defined  by  the  customer  and  the  organization’s  acquisition  staff  acts  as  a 
channel  member  between  the  supplier  and  the  customer.  This  model  is  presented  in 
Figure  5. 
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Figure  5.  Proposed  B2B  Gaps  Model  for  Customer-Defined  Services 
D.  REGULATORY  AND  STATUTORY  COMPLIANCE 

The  government’s  purchasers,  administrators,  and  inspectors  of  services 
acquisitions  are  tasked  with  ensuring  compliance  to  the  multitude  of  federal  regulations 
and  statutes  as  well  as  to  any  additional  regulations  or  procedures  that  may  be  required  by 
the  procuring  agency  in  supplements  to  the  Federal  Acquisition  Regulation  (FAR), 
collectively  referred  to  as  the  Federal  Acquisition  System  (FAR,  2005;  Riddell,  1985). 
For  U.S.  Air  Force  acquisition  personnel,  the  Federal  Acquisition  System  includes  the 
Federal  Acquisition  Regulation,  the  Defense  Federal  Acquisition  Regulation  Supplement 
(DFARS),  and  the  Air  Force  Federal  Acquisition  Regulation  Supplement  (AFFARS). 
Additional  acquisition  regulations  that  are  outside  of  the  Federal  Acquisition  System  may 
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include  agency  guidance,  mandatory  procedures,  policy  letters,  and  case  law.  A  review 
of  FAR  1.102  states  that  the  use  of  the  Federal  Acquisition  System  will  satisfy  the 
customer  in  terms  of  quality,  among  other  objectives,  as  a  guiding  principle. 

However,  the  U.S.  Air  Force  and  the  DoD  have  come  under  much  scrutiny 
regarding  compliance  with  procurement  regulations  and  statutes  (see,  for  example,  GAO, 
2000,  2005a,  2005c,  2007a).  Not  all  cases  of  non-compliance  are  due  to  fraud,  waste,  or 
abuse;  in  fact,  many  cases  are  simply  the  result  of  unknowledgeable  and  inadequately 
trained  personnel — an  effect  of  large-scale  increases  in  agency  contracting  without 
commensurate  changes  to  hiring  or  training  practices,  termed  -eorruption  by 
incompetence”  by  Cohen  and  Eimicke  (2008,  p.  30).  Nonetheless,  recent  statistics  about 
procurement  fraud  have  been  astonishing:  DoD  personnel  alone  accounted  for  718 
criminal  indictments  and  565  criminal  convictions  in  procurement  fraud  cases  between 
fiscal  years  2001  and  2005  (GAO,  2006a).  Whether  caused  by  poor  ethics,  incompetence, 
or  other  factors,  non-compliance  with  regulations  and  statutes  undermines  acquisition 
policy  objectives  and  often  thwarts  those  procedures  established  to  ensure  that  the 
government  receives  the  best  value  for  the  taxpayer  dollar.  Therefore,  it  is  hypothesized 
that, 


Hypothesis  1:  The  greater  the  compliance  with  regulations  and  statutes,  the 
greater  the  service  quality. 

E.  REQUIREMENT  DEFINITION 

Purchasers  of  goods  and  services  must  clearly  define  contractual  requirements  in 
order  to  properly  achieve  acquisition  objectives,  as  is  evident  in  the  gaps  model  that  is 
proposed  in  this  thesis  for  customer-defined  services  (Figure  5)  and  in  the  gaps  models 
developed  by  Parasuraman  et  al.  (1985)  and  Kong  and  Mayo  (1993).  In  these  models, 
one  or  more  gaps  exist  between  the  buyer’s  expectations  of  the  service  and  the  supplier’s 
interpretation  of  the  buyer’s  expectations.  Because  the  supplier’s  interpretation  of  the 
buyer’s  expectations  ultimately  affects  the  level  of  service  perfonnance,  the  buyer’s 
perception  of  the  quality  of  the  actual  service  delivery  is  largely  dependent  upon  how 
sufficiently  the  buyer’s  expectations  and  the  supplier’s  interpretation  of  those 
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expectations  (expectations  and  discontinuation)  align.  Buyers  of  customer-defined  B2B 
services  often  state  service  expectations  in  requirements  documents  that  are  incorporated 
into  purchase  agreements  or  contracts. 

Within  public  procurement,  10  U.S.C.  §  2305  requires  purchasers  to  state 
government  specifications  in  terms  of  function,  performance,  or  design  requirements. 
However,  when  acquiring  services,  specification  and  measurement  of  the  required  level 
of  quality  is  often  more  complex  than  when  acquiring  goods  (Brynste,  1996).  This 
dichotomy  is  prevalent  within  the  DoD  where  recent  reports  have  highlighted  several 
instances  of  decreased  acquisition  outcomes  due  to  insufficiently  defined  service 
requirements  (GAO  2002b,  2007b,  2009b).  To  address  some  of  these  issues,  the 
USD(AT&L)  has  recently  called  for  the  strengthening  of  services  requirements 
documents  through  the  use  of  standardized  work  statements  and  the  establishment  of 
market  research  teams  at  the  portfolio  management  level  (USD[AT&L],  2010b).  Even 
with  these  additional  tools,  the  added  complexity  of  differing  interpretations  of 
requirements  documents  by  the  buyer  and  supplier  renders  the  exact  communication  of 
the  contents  of  the  service  and  the  desired  service  level  to  be  nearly  impossible.  Without 
a  complete  understanding  of  the  buyer’s  service  requirement,  a  supplier  may  not  perform 
tasks  that  the  government  expects  to  receive  under  the  terms  of  the  contract;  may  not 
meet  the  buyer’s  expectations  in  terms  of  function,  performance,  and  quality;  or  may  not 
perfonn  those  functions  necessary  for  the  contract  to  adhere  to  regulations,  statutes,  and 
policy.  Furthermore,  because  the  requirement  definition  represents  the  buyer’s 
expectations  of  service  levels,  and  the  supplier’s  perfonnance  is  often  largely  dependent 
on  interpretation  of  the  requirements  documents,  the  buyer’s  perception  of  service  quality 
is  intrinsically  linked  to  the  sufficiency  with  which  requirements  are  defined  and 
communicated.  Therefore, 

Hypothesis  2:  There  will  be  a  direct,  positive  relationship  between  the  sufficiency 
of  the  requirement  definition  and  service  quality. 

Hypothesis  3:  There  will  be  a  direct,  positive  relationship  between  the  sufficiency 
of  the  requirement  definition  and  regulatory  and  statutory  compliance. 
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Recent  literature  has  indicated  that  an  early  involvement  of  suppliers  into  supply 
chains  may  produce  positive  outcomes  for  both  buyers  and  suppliers  (see,  for  example, 
Handheld,  Ragatz,  Petersen,  &  Monczka,  1999;  Seshadri,  2005).  Briscoe,  Dainty, 
Millett,  and  Neale  (2004)  examined  strategies  for  the  improvement  of  construction  supply 
chains  and  found  that  the  early  involvement  of  suppliers  resulted  in  increased  integration 
into  the  supply  chain,  improvements  to  schedule,  and  a  better  understanding  of  client 
needs  and  objectives,  among  other  benefits.  With  respect  to  services  acquisitions, 
Briscoe  et  al.’s  (2004)  conclusion  represents  a  logical  outcome  as  purchasing 
organizations  step  away  from  an  introverted  approach  to  procurement  planning  and 
capitalize  on  the  expertise  of  suppliers  that  are  often  more  experienced  and 
knowledgeable  in  their  respective  industries  than  the  purchasing  organization’s  buyers. 
As  such, 

Hypothesis  4:  The  greater  the  extent  to  which  the  contractor  defines 

requirements,  the  greater  the  service  quality. 

Hypothesis  5:  The  greater  the  extent  to  which  the  contractor  defines 

requirements,  the  greater  the  sufficiency  of  the  requirement  definition. 

F.  RELATIONAL  EXCHANGE 

Relational  contract  theory,  also  called  essential  contract  theory,  was  introduced  by 
Macneil  (1980),  who  contended  that  relationships,  rather  than  discrete  transactions,  are  at 
the  core  of  contracts.  Discrete  transactions  are  of  a  short  duration,  have  a  definite 
beginning  and  end,  and  involve  anonymous  parties.  Conversely,  relationships  are  a 
sequence  of  exchanges  between  parties  that  are  known  to  each  other,  typically  lasting  a 
long  period  of  time  or  indefinitely  (Dwyer,  Schurr,  &  Oh,  1987).  Macneil  (1980) 

developed  a  list  of  10  norms,  or  expectations  of  behavior,  that  are  common  to  all 

contracts  to  some  degree  and  that  would  increase  contractual  benefits  for  both  parties  if 
present.  These  norms,  which  are  at  least  partly  shared  by  all  members  in  a  transaction, 
largely  govern  individual  exchange  relationships  between  firms  (Heide  &  John,  1992). 
The  10  norms  are  as  follows: 
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1 .  role  integrity; 

2.  reciprocity; 

3.  implementation  of  planning; 

4.  effectuation  of  consent; 

5.  flexibility; 

6.  contractual  solidarity; 

7.  restitution,  reliance,  and  expectation  interests; 

8.  creation  and  restraint  of  power; 

9.  propriety  of  means;  and 

10.  harmonization  with  the  social  matrix. 

In  an  extension  on  relational  contract  theory,  Morgan  and  Hunt  (1994)  developed 
the  commitment-trust  theory  for  relational  marketing.  In  their  model,  commitment  and 
trust  are  central  because  they  encourage  marketers  to  work  toward  preserving 
investments  in  relationships  through  cooperation;  favor  beneficial  long-term  partnerships 
over  short-term,  volatile  pacts;  and  consider  higher-risk  endeavors  without  fear  of 
partners  acting  in  opportunistic  manners  (Morgan  &  Hunt,  1994).  Commitment  is  a  long¬ 
term  desire  to  maintain  a  relationship  that  is  considered  to  be  important  or  valuable,  and 
trust  is  a  reflection  of  willingness  to  depend  on  a  business  partner.  In  Morgan  and  Hunt’s 
model,  communication  is  a  precursor  to  trust,  which  ultimately  results  in  successful 
relational  exchanges  between  parties  by  providing  a  mechanism  for  partners  to  resolve 
disputes,  align  their  expectations  and  perceptions,  and  jointly  develop  strategies 
(Wittmann,  Hunt,  &  Arnett,  2009). 

Two  of  the  procurement  theories  that  explain  firm  governance  are  relational 
exchange  and  transaction  cost  economics  (TCE).  TCE  theory  suggests  that  activities  will 
be  outsourced  when  transaction  costs  are  lower  than  the  cost  of  perfonning  the  work  in- 
house  (Williamson,  1975).  The  transaction  costs  of  contracting  in  the  market  include  the 
costs  of  writing  and  negotiating  contracts  and  the  costs  of  monitoring  suppliers — actions 
needed  to  thwart  supplier  opportunism.  Relational  exchange  offers  a  more  efficient 
alternative  to  governing  suppliers.  By  establishing  trust  and  commitment,  suppliers  need 
not  be  monitored  as  closely  and  the  contract  need  not  be  written  as  thoroughly.  After  all, 
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even  the  most  thorough  contracts  cannot  possibly  cover  all  contingencies.  Numerous 
positive  effects  of  relational  exchange  include  increased  cooperation  (Morgan  &  Hunt, 
1994,  reduced  opportunism  (Brown,  Dev,  &  Lee,  2000;  Joshi  &  Stump,  1999),  increased 
performance  (Skarmeas,  Katsikeas,  &  Schlegelmilch,  2002),  and  increased  satisfaction 
and  service  quality  (Paulin,  Perrien,  &  Ferguson,  1997).  Therefore,  given  the  centrality 
of  relational  exchange  or  buyer-supplier  dynamics,  any  study  of  B2B  exchange  should 
include  the  effects  of  relational  norms. 

When  contracting  for  services,  proper  communication  between  a  services 
purchaser,  contractor,  and  end  user  is  critically  important  to  handle  variations  or 
unforeseen  events  in  service  delivery  (Bryntse,  1996).  Cohen  and  Eimicke  (2008) 
included  several  forms  of  government-contractor  communication  problems  among  their 
list  of  top  issues  within  public  procurement.  First,  they  argued  that  poor  communication 
between  the  government  and  the  contractor’s  management  team  often  produces  an 
unacceptable  level  of  performance  and  causes  conflicts  between  the  parties,  particularly 
when  communication  issues  result  in  poorly  defined  tasks  or  when  projects  fail. 
Similarly,  they  reasoned  that  communication  issues  between  the  government  and  the 
contractor’s  employees  at  the  staff  level  may  result  in  employee  confusion  regarding 
direction  on  tasks  or  assignments.  In  these  situations,  the  authors  argued  that  the 
contractor’s  staff-level  employees  are  often  forced  to  establish  direction  and  solve 
problems  internally,  which  may  result  in  methods  or  levels  of  performance  that  do  not 
match  the  government’s  expectations.  In  parallel  with  this  line  of  reasoning,  inadequate 
communication  between  the  government  and  the  contractor’s  managerial  or  staff-level 
employees  may  result  in  undesirable  contractor  performance  on  those  actions  required  to 
ensure  compliance  with  the  government’s  procurement  policies.  Kong  and  Mayo  (1993) 
reiterated  the  importance  of  this  function-level  communication.  They  emphasized  the 
need  for  supply  chain  members  to  integrate  (e.g.,  through  high  involvement  and 
frequency  of  contact)  the  respective  functional  areas  of  each  firm  in  order  to  minimize 
gaps  and  maximize  service  levels  to  the  end  consumer.  Likewise,  they  also  warned  that 
where  buyer-supplier  interfaces  are  constrained  (e.g.,  because  cross-functional,  cross- 
organizational  dialogue  is  controlled  or  stymied),  gaps  in  service  delivery  will  occur. 
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This  is  logical — particularly  when  requirements  documents  inadequately  define 
expectations  and  needs.  Where  specifications  are  vague,  communication  can  fill  the 
void.  As  such, 

Hypothesis  6:  The  greater  the  communication  between  the  government  and  the 
contractor,  the  greater  the  service  quality. 

Hypothesis  7:  The  greater  the  communication  between  the  government  and  the 
contractor,  the  greater  the  regulatory  and  statutory  compliance. 

G.  AGENCY  THEORY 

In  agency  theory,  the  agency  relationship  is  defined  as  an  agreement  in  which  at 
least  one  person,  the  principal,  delegates  duties  and  some  decision-making  authority  to 
another,  the  agent  (Jensen  &  Meckling,  1976).  Jensen  and  Meckling  (1976)  described  the 
agency  problem  as  the  likelihood  that  the  agent  will  not  act  in  the  interest  of  the  principal 
if  both  parties  seek  to  maximize  their  utility.  Eisenhardt  (1989)  asserted  that  two 
fundamental  problems  may  occur  in  principal-agent  relationships.  The  first  problem  will 
ensue  when  the  principal  and  agent  have  conflicting  goals,  and  it  is  either  difficult  or 
expensive  for  the  principal  to  monitor  the  agent.  The  second  problem  occurs  when  the 
principal  and  agent  have  different  attitudes  toward  accepting  risk.  The  principal  can  limit 
actions  by  the  agent  that  are  misaligned  with  the  principal’s  interests  by  expending 
additional  resources  on  monitoring  the  agent,  by  offering  incentives,  or  by  paying  for  the 
agent’s  bonding.  Therefore,  the  cost  of  the  agency  relationship,  or  the  agency  cost,  is  the 
sum  of  these  three  actions  by  the  principal  (Jensen  &  Meckling,  1976). 

Within  public  procurement,  a  misalignment  of  interests  between  the  government 
(principal)  and  a  contractor  (agent)  is  typically  identified  through  the  use  of  surveillance 
methods,  often  termed  quality  assurance.  Lam  (2008)  asserted  that  surveillance  is 
necessary  to  ensure  service  quality  in  public  procurements  and  argued  for  its 
effectiveness.  Other  authors  tend  to  agree;  Axelsson  and  Wynstra  (2002)  wrote  that 
service  quality  is  as  dependent  on  the  post-award  management  of  performance  as  it  is  on 
the  pre-award  specification  of  service  requirements  and  source  selection.  Despite  this, 
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the  GAO  found  that  contracting  activities  within  the  DoD  have  typically  placed  a  far 
greater  emphasis  on  the  act  of  awarding  service  contracts  than  on  ensuring  that  trained 
and  knowledgeable  quality  assurance  personnel  are  assigned  prior  to  contract  award 
(GAO,  2005b).  The  GAO  also  reported  that  these  actions,  and  others  that  lead  to 
inadequate  post-award  surveillance,  reduce  the  government’s  ability  to  assure  that  service 
suppliers  are  providing  timely  and  quality  services  and  mitigating  performance  problems. 
Therefore, 

Hypothesis  8:  The  greater  the  amount  of  government  surveillance  of  contractor 

performance,  the  greater  the  service  quality. 

The  federal  government  and  the  DoD  prescribe  the  implementation  of 
perfonnance-based  services  acquisition  (PBSA)  procedures  in  order  to  address  agency 
problems  that  occur  in  the  acquisition  of  services.  Most  notably,  PBSA  promotes  the 
procurement  of  commercial  services  and  promotes  contractor  innovations  through  the  use 
of  outcome-based  requirement  definitions,  as  opposed  to  requirement  definitions  that 
specify  the  inputs  and  tasks  necessary  for  performance  (USD[AT&L],  2000).  The  OFPP 
enumerated  four  requirements  that,  at  a  minimum,  must  be  included  in  a  service  contract 
for  the  proper  implementation  of  PBSA  procedures:  an  outcome-based  requirement 
definition,  perfonnance  standards  that  are  tied  to  requirements,  a  government-developed 
plan  for  monitoring  contractor  perfonnance  against  performance  standards,  and,  when 
appropriate,  positive  and  negative  performance  incentives  (Office  of  Federal  Procurement 
Policy  [OFPP],  1997).  Although  the  DoD  has  had  some  issues  fully  implementing  PBSA 
procedures  (see,  for  example,  Ausink,  Baldwin,  Hunter,  &  Shirley,  2002;  GAO,  2002b), 
the  use  of  PBSA  in  public  procurement  has  been  linked  to  improved  acquisition 
outcomes — most  notably,  reduced  cost  and  improved  performance  (OFPP,  1998).  As 
such, 


Hypothesis  9:  The  greater  the  extent  to  which  performance-based  services 
acquisition  procedures  are  used,  the  greater  the  service  quality. 
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H.  RESOURCE-BASED  VIEW 


The  resource-based  view  of  the  firm  states  that  a  firm’s  competitive  advantage  in 
the  marketplace  is  based  on  its  ability  to  acquire  and  maintain  valuable  resources 
important  to  production  (Connor,  1991).  Resources  are  a  firm’s  physical  capital,  human 
capital,  and  organizational  capital  that  improve  efficiency  or  effectiveness  (Barney, 
1991).  However,  not  all  of  a  firm’s  capital  should  be  considered  a  resource  because  some 
capital  could,  in  fact,  reduce  efficiency  or  effectiveness. 

For  a  firm  to  build  a  sustained  competitive  advantage,  defined  as  a  competitive 
advantage  that  lasts  a  long  period  of  time  (Jacobsen,  1988),  firms  must  possess  resources 
that  are  rare,  valuable,  imperfectly  imitable,  and  non-substitutable  by  other  resources  that 
are  valuable  but  neither  rare  nor  imperfectly  imitable  (Barney,  1991).  Barney  (1991) 
asserted  that  no  firm  can  gain  a  competitive  advantage  in  an  industry  in  which  firms 
possess  exactly  the  same  resources  (homogeneity)  and  in  which  the  resources  are 
perfectly  mobile.  However,  for  a  firm  that  possesses  a  competitive  advantage  to  achieve 
returns  that  are  above  normal,  it  must  either  produce  a  product  that  is  distinctive  in  the 
eyes  of  buyers  or  sell  a  product  that  is  comparable  to  that  of  other  firms  but  at  a  lower 
cost  (Connor,  1991;  Porter,  1985). 

In  terms  of  the  acquisition  of  services  in  the  U.S.  Air  Force,  alliance  resources 
between  the  purchasing  activity  and  the  requiring  activity  include  acquisition  personnel 
and  the  time  allotted  for  those  personnel  to  perform  all  of  the  functions  necessary  for  the 
acquisition  of  the  service.  First,  the  sufficiency  of  the  procurement  lead-time,  or  time 
available  to  perfonn  those  contractual  functions  required  prior  to  award,  varies  between 
procurements  and  is  dictated  by  the  period  between  the  purchaser  learning  of  the 
requirement  and  the  contract  being  awarded.  The  length  of  this  period  may  determine 
how  well  the  requirement  is  defined,  the  amount  or  depth  of  market  research  that  is 
perfonned,  the  appropriateness  of  the  acquisition  strategy,  and  the  ability  of  the 
contracting  activity  to  comply  with  applicable  directives  such  as  advertising 
requirements,  competing  requirements,  applying  appropriate  socio-economic  strategies 
(e.g.,  set-asides),  documenting  determinations  and  findings,  and  conducting  solicitation 
and  contract  reviews.  As  such,  it  is  posited  that, 
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Hypothesis  10:  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater 
the  service  quality. 

Hypothesis  11:  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater 
the  compliance  with  regulations  and  statutes. 

Hypothesis  12:  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater 
the  sufficiency  of  the  requirement  definition. 

As  a  resource  in  the  purchaser-supplier  alliance,  the  assignment  of  an  adequate 
number  of  personnel  to  perfonn  contract  award  and  administration  functions  is  crucial  to 
the  acquisition’s  overall  success.  Recent  reports  by  the  DoD  Inspector  General 
highlighted  issues  resulting  from  insufficient  manpower  and  increased  turnover  due  to 
acquisition  workforce  reductions  (DoDIG,  2000a,  2000b,  2003).  Several  key  areas  in 
which  problems  were  noted  include  increased  program  costs,  reduced  scrutiny  and 
timeliness  in  reviewing  acquisition  actions,  lost  opportunities  to  develop  cost-saving 
initiatives,  insufficient  staff  to  manage  requirements,  and  increased  backlogs  in  closing 
out  completed  contracts  (DoDIG,  2000a).  In  particular,  the  excessive  turnover  of 
acquisition  personnel  threatens  the  long-tenn  success  of  acquisitions  as  government 
administration  functions  that  are  required  by  contract  terms,  regulations,  or  statutes  may 
not  be  properly  accomplished  (Special  Inspector  General  for  Iraq  Reconstruction 
[SIGIR],  2008). 

Dalton,  Krackhardt,  and  Porter  (1981)  offered  multiple  classifications  for 
employee  turnover,  noting  that  some  forms  of  turnover  may  not  be  harmful  to 
organizations.  For  example,  organizations  may  benefit  when  lower-performing  or 
disruptive  employees  depart  (Mathis  &  Jackson,  2003).  Other  forms  of  turnover,  such  as 
the  departure  of  high-performing  personnel  at  critical  times,  are  clearly  detrimental  to  an 
organization’s  ability  to  achieve  desired  objectives.  As  such,  some  authors  have  argued 
that  an  optimal  level  of  employee  turnover  exists  for  firms  (see,  for  example,  Abelson  & 
Baysinger,  1984).  Other  authors  broadly  contend  that  turnover  is  associated  with 
decreased  organizational  effectiveness  (Glebbeek  &  Bax,  2004).  Despite  these  differing 
views,  research  has  generally  supported  the  deleterious  effects  of  high  levels  of  employee 
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turnover  on  organizational  performance  (see,  for  example,  Glebbeek  &  Bax,  2004; 
Huselid,  1995),  regardless  of  turnover  cause  or  classification.  Schlesinger  and  Heskett 
(1991)  asserted  that  high  employee  turnover  in  service  organizations  results  in  long-tenn 
decreases  in  sales  and  profitability  as  well  as  in  lower  levels  of  service  quality.  Harrison 
(2008)  noted  that  high  levels  of  employee  turnover  in  the  aerospace  and  defense  industry 
have  created  a  knowledge  gap;  vital  infonnation  has  left  companies  along  with  their 
employees.  As  such,  increased  employee  turnover  is  generally  associated  with  decreased 
efficiency  and  a  diminished  ability  to  meet  organizational  objectives,  especially  when  the 
level  of  turnover  is  high  or  excessive.  Therefore, 

Hypothesis  13:  The  greater  the  turnover  of  government  acquisition  personnel,  the 
lesser  the  service  quality. 

Hypothesis  14:  The  greater  the  turnover  of  government  acquisition  personnel,  the 
lesser  the  compliance  with  regulations  and  statutes. 

I.  COMPETENCE-BASED  VIEW 

Often  considered  an  extension  to  the  resource-based  view,  the  competence-based 
view  claims  that  competitive  advantage  is  a  function  of  a  firm’s  core  competencies. 
Sanchez,  Heene,  and  Thomas  (1996)  defined  competence  as  the  ability  to  sustain  the 
coordinated  deployment  of  assets  in  ways  that  help  a  firm  achieve  its  goals.  Prahalad  and 
Hamel  (1990)  applied  a  three-part  litmus  test  to  identify  core  competences  in  a  firm. 
First,  a  core  competency  should  offer  potential  access  to  a  range  of  markets,  such  as 
Intel’s  core  competency  of  manufacturing  semiconductors.  Semiconductors  have  a  broad 
range  of  uses  in  multiple  markets  from  consumer  electronics — such  as  computers, 
televisions,  radios,  and  phones — to  the  high-tech  worlds  of  aerospace  and  medicine. 
Second,  a  core  competency  should  provide  a  benefit  to  the  consumer;  such  a  benefit  may 
be  perceived  or  actual.  Intel’s  corporate  experience  and  status  as  a  frontrunner  in  the 
semiconductor  industry  are  likely  viewed  as  benefits  by  many  of  the  company’s 
customers.  Finally,  the  competency  should  be  difficult  for  competitors  to  imitate,  such  as 
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Intel’s  implementation  of  simultaneous  multithreading  into  the  central  processing  units  of 
personal  computers. 

Econom  (2006)  argued  that  federal  agencies  must  consider  contract  management 
as  a  core  competency  because  the  functions  performed  by  third-party  contractors  are 
often  essential  in  successfully  achieving  organizational  goals,  and  she  concluded  that  the 
success  of  acquisition  organizations  is  largely  dependent  on  hiring  personnel  who  possess 
the  right  mix  of  skills,  abilities,  experience,  and  training.  Other  studies  have  also  found 
that  this  right  mix  is  critical  to  achieving  contract  performance  outcomes  (United  States 
Merit  Systems  Protection  Board,  2005).  Within  services  acquisition,  personnel 
education,  training,  and  experience  are  enablers  for  the  purchasing  organization  to 
effectively  deploy  assets,  monetary  and  otherwise,  to  achieve  acquisition  objectives. 
Those  individuals  with  the  greatest  breadth  of  education,  training,  and  experience  may  be 
capable  of  effectively  purchasing  and  administering  a  wider  range  of  service  contracts  to 
meet  customer  requirements.  Although  the  development  of  knowledge  may  be  a  result  of 
broad-based  practical  and  educational  exposure,  experience  is  often  a  function  of  time 
spent  perfonning  tasks.  Purchaser  education  typically  occurs  outside  of  the  work 
environment  and  may  be  reflected  by  the  granting  of  degrees  or  by  professional 
certifications.  The  level  of  training  of  the  federal  acquisition  workforce  is  measured 
using  the  Acquisition  Professional  Development  Program  (APDP),  which  offers  three 
levels  of  certification  in  several  areas  of  acquisition  based  on  the  completion  of  training 
courses  and  modules,  on-the-job  experience  (time),  and  the  applicant’s  level  of  education. 
In  practice,  federal  acquisition  personnel  who  demonstrate  a  capability  for  increased 
responsibility  through  competencies  of  education,  training,  and  work  experience  may  be 
assigned  to  award  or  administer  acquisitions  that  are  more  complex  in  definition  or 
structure,  requiring  compliance  with  increased  numbers  of  regulations  and  statutes.  As 
such, 


Hypothesis  15:  There  will  be  a  direct,  positive  relationship  between  the  amount  of 
contract  administrator  experience  and  the  sufficiency  of  the  requirement 
definition. 
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Hypothesis  16:  There  will  be  a  direct,  positive  relationship  between  the  amount  of 
contract  administrator  experience  and  regulatory  and  statutory  compliance. 

Hypothesis  17:  The  greater  the  contract  administrator’s  APDP  certification  level, 
the  greater  the  sufficiency  of  the  requirement  definition. 

Hypothesis  18:  The  greater  the  contract  administrator’s  APDP  certification  level, 
the  greater  the  regulatory  and  statutoiy  compliance. 

Hypothesis  19:  The  greater  the  contract  administrator’s  education  level,  the 
greater  the  sufficiency  of  the  requirement  definition. 

Hypothesis  20:  The  greater  the  contract  administrator’s  education  level,  the 
greater  the  regulatory  and  statutory  compliance. 

J.  INTERNAL  CUSTOMER  COMMITMENT 

The  many  roles  of  the  client,  or  internal  customer,  are  critical  to  the  success  of  a 
supply  chain  throughout  the  life  cycle.  Briscoe  et  al.  (2004)  found  that  the  internal 
customer’s  desire  to  develop  supply  chain  relationships  was  the  single  most  important 
factor  to  achieving  supply  chain  integration.  The  authors  also  found  that,  during  the 
requirements  definition  stage  of  procurement,  client  organizations  influenced  how  the 
project  team  was  shaped  and  the  selection  of  the  procurement  method.  Similarly,  the 
many  roles  of  internal  customers  in  U.S.  Air  Force  services  acquisition  are  critical  to 
overall  acquisition  success. 

First,  the  internal  customer  often  provides  the  necessary  funding  to  acquire  the 
service.  This  level  of  funding  may  pennit  the  use  of  certain  performance-based 
incentives,  if  appropriate  for  the  contractual  action.  Second,  the  internal  customer  plays 
an  integral  role  in  the  generation  of  requirements  documents,  such  as  the  Statement  of 
Work  or  the  Performance  Work  Statement,  as  well  as  in  the  creation  of  other  documents 
that  may  be  necessary  for  the  proper  execution  or  surveillance  of  the  acquisition.  Third, 
the  internal  customer  must  devote  manpower  to  the  services  acquisition  for  (1)  the 
evaluation  of  offers  throughout  the  source  selection  process,  (2)  the  performance  of 
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quality-assurance  functions  that  are  necessary  to  the  surveillance  of  contractor 
perfonnance,  and  (3)  the  management  of  the  requirement,  including  contract-change 
requests,  development  of  past  performance  information,  and  the  determination  of 
perfonnance-based  incentive  awards.  This  level  of  manpower  is  often  based  on 
manpower  standards  that  allocate  full-time  positions  based  on  the  number  of  contracts 
that  a  unit  manages,  among  other  factors  (Reed,  2010;  U.S.  Air  Force,  2001).  In  such  a 
situation,  the  assignment  of  available  personnel  to  contracts  within  a  unit’s  portfolio 
would  be  an  indication  of  the  level  of  commitment  to  each  acquisition.  Without  an 
acceptable  level  of  commitment  from  the  internal  customer  to  properly  conduct  the 
acquisition,  adequate  standards  for  the  level  of  service  quality  may  not  be  established  and 
the  contract  may  not  be  effectively  managed  after  award  to  ensure  that  contractual, 
regulatory,  and  statutory  requirements  are  met.  As  such, 

Hypothesis  21:  The  greater  the  internal  customer’s  commitment  to  the  service 
acquisition,  the  greater  the  compliance  with  regulations  and  statutes. 

Hypothesis  22:  The  greater  the  internal  customer’s  commitment  to  the  service 
acquisition,  the  greater  the  sufficiency  of  the  requirement  definition. 

K.  CONCLUSION 

In  this  chapter,  a  total  of  22  relationships  were  hypothesized  based  on  a  review  of 
relevant  literature  and  theory;  a  summary  of  these  hypotheses  can  be  found  in  Table  1. 
The  following  chapter  presents  the  research  and  statistical  methods  used  to  explore  these 
relationships,  along  with  measurement  scales  and  a  path  diagram  of  the  conceptualized 
modeling  of  the  determinants  of  services  sourcing  performance. 
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Table  1.  Research  Hypotheses 


Notation 


Hypothesis 


HI  The  greater  the  compliance  with  regulations  and  statutes,  the  greater  the  service  quality. 

H2  There  will  be  a  direct,  positive  relationship  between  the  sufficiency  of  the  requirement  definition 
and  service  quality. 

H3  There  will  be  a  direct,  positive  relationship  between  the  sufficiency  of  the  requirement  definition 
and  regulatory  and  statutory  compliance. 

H4  The  greater  the  extent  to  which  the  contractor  defines  requirements,  the  greater  the  service 
quality. 

H5  The  greater  the  extent  to  which  the  contractor  defines  requirements,  the  greater  the  sufficiency  of 
the  requirement  definition. 

H6  The  greater  the  communication  between  the  government  and  the  contractor,  the  greater  the 
service  quality. 

H7  The  greater  the  communication  between  the  government  and  the  contractor,  the  greater  the 
regulatory  and  statutory  compliance. 

H8  The  greater  the  amount  of  government  surveillance  of  contractor  perfonnance,  the  greater  the 
service  quality. 

H9  The  greater  the  extent  to  which  performance-based  services  acquisition  procedures  are  used,  the 
greater  the  service  quality. 

H10  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater  the  service  quality. 

H 1 1  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater  the  compliance  with 
regulations  and  statutes. 

H12  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater  the  sufficiency  of  the 
requirement  definition. 

HI  3  The  greater  the  turnover  of  government  acquisition  personnel,  the  lesser  the  service  quality. 

H14  The  greater  the  turnover  of  government  acquisition  personnel,  the  lesser  the  compliance  with 

regulations  and  statutes. 

HI 5  There  will  be  a  direct,  positive  relationship  between  the  amount  of  contract  administrator 
experience  and  the  sufficiency  of  the  requirement  definition. 

HI 6  There  will  be  a  direct,  positive  relationship  between  the  amount  of  contract  administrator 
experience  and  regulatory  and  statutory  compliance. 

HI 7  The  greater  the  contract  administrator’s  APDP  certification  level,  the  greater  the  sufficiency  of 
the  requirement  definition. 

HI 8  The  greater  the  contractor  administrator’s  APDP  certification  level,  the  greater  the  regulatory 
and  statutory  compliance. 

H19  The  greater  the  contract  administrator’s  education  level,  the  greater  the  sufficiency  of  the 
requirement  definition. 

H20  The  greater  the  contract  administrator’s  education  level,  the  greater  the  regulatory  and  statutory 
compliance. 

H21  The  greater  the  internal  customer’s  commitment  to  the  service  acquisition,  the  greater  the 
compliance  with  regulations  and  statutes. 

H22  The  greater  the  internal  customer’s  commitment  to  the  service  acquisition,  the  greater  the 
sufficiency  of  the  requirement  definition. 
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III.  METHODOLOGY 


A.  INTRODUCTION 

This  chapter  will  first  discuss  the  design  of  the  research  study,  including  the 
statistical  methodology  and  the  data  collection  methodology  selected.  A  conceptual 
model  is  then  presented  in  which  all  variables  and  hypothesized  relationships  are  visually 
depicted  in  a  path  diagram.  Finally,  the  population  sampled  in  the  study  is  discussed, 
followed  by  a  presentation  of  the  measurement  scales  for  constructs  and  manifest 
variables. 

B.  RESEARCH  DESIGN 

This  study  seeks  to  precisely  identify  and  measure  the  determinants  of  services 
sourcing  perfonnance  through  the  testing  of  models  and  hypotheses.  As  a  result,  the 
research  lends  itself  to  a  quantitative  approach  because  hypothesized  relationships  have 
been  identified  and  because  the  scales  for  measurement  of  these  hypothesized 
relationships  already  exist,  can  be  adapted  from  previous  research,  or  can  be  created. 
Quantitative  analysis  is  an  explanation  of  phenomena,  in  numerical  terms,  through 
statistical  and  mathematical  testing  of  numerical  data.  This  study  will  primarily  use 
Structural  Equation  Modeling  (SEM)  for  statistical  analysis — a  method  chosen  due  to  its 
numerous  advantages  over  other  common  statistical  techniques. 

SEM  is  a  multivariate  statistical  method  used  to  conduct  confirmatory  factor 
analysis  and  to  test  causal  hypothesized  relationships  between  manifest  and  unobserved 
(latent)  variables.  Latent  variables  are  those  variables  that  are  not  directly  observable  but 
that  are  inferred  from  a  set  of  other  observed  variables  and,  when  utilizing  a  survey  for 
data  collection,  are  often  measured  using  a  scale.  In  SEM,  latent  or  observed  variables 
that  are  unexplained  in  a  model  are  considered  to  be  exogenous,  having  one  or  more 
causal  paths  that  lead  from  them  to  other  variables.  Conversely,  variables  that  are 
explained  in  the  model  are  considered  to  be  endogenous  and  have  one  or  more  causal 
paths  leading  to  them  from  other  endogenous  or  exogenous  variables. 
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At  the  most  fundamental  level,  SEM  provides  researchers  a  tool  to  measure 
multivariate  relationships  containing  a  mixture  of  latent  constructs  and  observed  variables 
(MacCallum,  1995).  One  primary  advantage  lies  in  the  ability  of  SEM  to  simultaneously 
evaluate  all  relationships  in  a  research  model  (Hair,  Black,  Babin,  &  Anderson,  2010). 
Additionally,  unlike  other  multivariate  techniques,  SEM  provides  an  assessment  of 
construct  measurement  error;  alternative  statistical  tools  such  as  regression  analysis 
typically  assume  that  there  is  no  error  in  measured  variables,  which  may  subject  a 
researcher  to  the  interpretation  of  faulty  results  (Byrne,  2010).  Furthermore,  SEM  allows 
for  the  analysis  of  mediating  variables  and  can  perfonn  a  simultaneous  analysis  of 
multiple  dependent  variables,  which  may  be  critical  to  accurately  estimating  complex 
models  (Hoyle,  1995).  Finally,  SEM  can  be  used  to  assess  the  overall  level  of  model  fit 
for  adequacy  and  for  comparison  against  alternative,  competing  models  (Hoyle  &  Panter, 
1995). 

After  selection  of  the  statistical  method,  the  second  step  in  the  research  design 
was  the  development  of  a  conceptual  model  that  visually  depicted  the  hypothesized 
relationships  that  were  studied.  Thus,  the  path  diagram  of  the  conceptual  model  (Figure 
6)  contains  22  relationships  that  represent  the  22  hypotheses,  3  endogenous  latent 
variables,  4  exogenous  latent  variables,  and  6  exogenous  observed  variables.  In  this  path 
diagram,  latent  constructs  are  depicted  using  ellipses  and  observed  variables  are  depicted 
using  rectangles.  Hypothesized  relationships  between  variables  are  depicted  using 
arrows;  the  directionality  of  causal  paths  begins  from  a  variable  at  the  arrow’s  tail  and 
ends  at  a  variable  about  the  arrow’s  point.  These  arrows  are  annotated  with  a  plus  or 
minus  sign  to  reflect  the  expected  sign  of  the  relationship. 
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Figure  6.  Conceptual  Model 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


c. 


SAMPLE 


The  population  for  this  study  consisted  of  U.S.  Air  Force  civilian  and  military 
contracting  personnel  who  administer  service  contracts  within  the  United  States.  Several 
constraints  existed  for  surveying  this  population.  First,  because  resources  and  timing 
prohibited  data  collection  from  every  individual  in  the  population,  data  were  drawn  from 
a  sample  consisting  of  the  broadest  range  of  contract  administration  personnel  within  the 
U.S.  Air  Force  using  a  web-based  survey.  Second,  a  list  of  contact  information  for 
potential  respondents  did  not  exist  and  could  not  reasonably  be  created.  Due  to  the  costs 
required  to  mail  a  sufficient  quantity  of  surveys  to  contracting  units  and  the  resources 
required  to  manually  input  responses  into  a  dataset,  surveys  were  deployed  by  forwarding 
to  commanders  and  supervisors  at  contracting  units  an  e-mail  that  contained  a  link  to  an 
online  survey.  Finally,  the  population  was  limited  to  contracting  personnel  who 
administer  service  contracts  within  the  United  States  because  cultural  and  business 
norms,  as  well  as  standards  for  the  application  of  contracting  procedures,  may  differ  in 
other  countries. 

Efforts  were  made  to  ensure  that  the  sample  provided  an  accurate  representation 
of  the  population;  however,  the  possibility  exists  that  unknown  factors  may  have 
predisposed  the  selection  towards  a  biased  sample  (Kerlinger  &  Lee,  2000).  Tests  for 
bias  due  to  non-response  are  presented  in  Chapter  IV. 

D.  SURVEY  DESIGN  AND  CONSTRUCT  MEASUREMENT 

A  total  of  44  measures  were  used  to  capture  the  7  latent  constructs,  which 
underlie  these  measures.  While  some  scales  had  to  be  created,  existing,  proven  scales 
were  used  when  possible.  All  latent  constructs  were  measured  using  7-point  Likert-type 
scales. 

Hartline  and  Ferrell  (1996)  suggested  that  unresolved  issues  with  the 
measurement  of  service  quality  have  caused  the  proper  selection  of  construct  measures  to 
become  problematic.  The  five-dimension  SERVQUAL  scale  developed  by  Parasuraman 
et  al.  (1988)  has  been  criticized  for  its  use  of  gap  scores,  its  mixture  of  positively  and 
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negatively  worded  questions,  and  its  inability  to  permit  researchers  to  generalize  the 
service  quality  dimensions  to  multiple  contexts.  This  scale  consists  of  44  questions, 
measuring  ex  ante  service  expectations  and  ex  post  perceptions  of  actual  service  quality. 
Conversely,  Cronin  and  Taylor  (1992)  contended  that  a  performance-only  measure  of 
service  quality,  such  as  SERVPERF,  is  more  effective  and  includes  dimensions  that  can 
be  generalized  over  a  broader  range  of  services.  However,  neither  scale  has  been  widely 
adopted  as  a  single  measure  of  service  quality.  As  an  alternative,  Hartline  and  Ferrell 
(1996)  argued  for  the  appropriateness  of  modification  to  the  SERVQUAL  scale  so  that 
each  separate  set  of  22  questions  for  the  measurement  of  expectations  and  perceptions  is 
combined  into  one  question  each  (for  a  total  of  22)  through  modification  of  the  Likert 
scale.  Hartline  and  Ferrell  (1996)  contended  that  service  customers  will  respond  to 
questions  regarding  quality  using  expectancy-disconfirmation  without  the  need  for  such  a 
separation.  A  similar  approach  was  taken  in  this  study. 

The  five  dimensions  of  SERVQUAL  proposed  by  Parasuraman  et  al.  (1988)  were 
tangibles,  reliability,  responsiveness,  assurance,  and  empathy.  Reliability  and  assurance 
are  specifically  applicable  and  of  interest  to  the  study  of  B2B  services,  such  as  those 
services  acquired  through  service  contracts  by  the  U.S.  Air  Force.  However,  tangibles, 
responsiveness,  and  empathy  do  not  fit  within  a  B2B  context.  Tangibles  refers  to 
physical  evidence  of  the  service — specifically  the  appearance  of  the  supplier’s  facility; 
the  appearance  of  personnel,  tools,  or  equipment  used  in  the  service;  the  physical 
representations  of  the  service;  and  the  perceptions  of  other  customers  in  the  supplier’s 
facility.  In  a  B2B  context,  and  specifically  on  a  military  installation,  it  is  unlikely  that 
customers  would  be  exposed  to  the  tangibles  of  service  delivery  for  many  contracts. 
Similarly,  responsiveness  and  empathy  assume  that  the  purchaser  has  face-to-face  contact 
with  the  service  provider  during  service  delivery,  which  is  less  typical  with  the 
purchasing  of  business-to-business  services.  The  scales  for  reliability  and  assurance  were 
therefore  retained  for  inclusion  in  the  study,  and  the  scales  for  tangibles,  empathy,  and 
responsiveness  were  discarded. 

A  new  dimension,  responsiveness  to  requirements,  was  added  to  better  suit  the 
assessment  of  service  quality  in  customer-defined  B2B  acquisitions.  Kong  and  Mayo 
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(1993)  attempted  to  address  the  additional  gaps  required  to  adapt  the  Gaps  Model 
developed  by  Parasuraman  et  al.  (1985)  to  the  context  of  B2B  services  but  fell  short;  the 
five  dimensions  of  SERVQUAL  still  do  not  fit  within  their  framework  for  B2B  services. 
As  such,  a  new  scale  containing  an  additional  dimension  is  needed  to  accurately  and 
validly  measure  service  quality  in  customer-defined  B2B  service  acquisitions.  The  items 
in  this  new  dimension,  responsiveness  to  requirements,  address  the  additional  gaps  in  the 
proposed  gaps  model  (Figure  4)  by  measuring  how  well  perceived  service  levels  reflect 
expected  service  levels,  as  defined  in  customer-generated  requirements  documents.  To 
assess  the  content  validity  of  the  proposed  dimension,  items  were  reviewed  by  three 
academicians  in  business-  and  supply-chain-related  fields  and  were  later  pre-tested 
qualitatively  by  practitioners  at  three  different  buying  locations.  Acquisition  personnel  at 
each  of  these  locations  administered  portfolios  of  customer-defined  B2B  service 
acquisitions  ranging  in  type  from  low-complexity  grounds  maintenance  services  to  highly 
technical  engineering  support  and  research  and  development  services.  The  final 
dimension  consists  of  five  items  that,  like  the  remainder  of  the  SERVQUAL  dimensions, 
were  measured  on  a  7-point  Likert-type  scale,  with  anchors  of  much  worse  than  I 
expected  and  much  better  than  I  excepted.  This  final  scale  for  the  measurement  of 
service  quality  is  presented  in  Table  2. 
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Table  2.  Construct  Measurement  of  Service  Quality 


Scale  item  Survey  question 

SQa*  When  the  contractor’s  management  promises  to  do  something  by  a  certain  time, 
it  does  so. 

SQb*  When  you  have  problems,  the  contractor’s  management  is  sympathetic  and 
reassuring. 

SQc*  The  contractor’s  dependability. 

SQd*  The  contractor  provides  its  services  at  the  time  it  promises  to  do  so. 

SQe  *  The  contractor’s  keeping  of  its  records  accurately. 

SQf**  How  much  you  can  trust  employees  of  the  contractor. 

SQg**  How  much  you  feel  safe  in  your  interactions  with  the  contractor’s  employees. 

SQh**  How  polite  the  contractor’s  employees  are. 

SQi  **  The  adequacy  of  the  support  the  contractor’s  employees  receive  from  their 
company  to  do  their  jobs  well. 

SQj  ***  The  contractor  met  the  requirements  of  the  contract. 

SQk  *  *  *  The  contractor  satisfied  our  need. 

SQI***  The  contractor  performed  the  work  we  needed  it  to  do. 

SQm***  The  timeliness  of  the  contractor’s  work. 

SQn***  The  quality  of  the  contractor’s  work. 

*Reliability,  **Assurance,  ***Responsiveness  to  Requirements. 

No  appropriate,  previously  validated  scales  were  available  for  the  measurement  of 
regulatory  and  statutory  compliance.  Because  an  observable  measure  for  compliance  that 
cited  specific  regulations  and  statutes  could  not  be  generalized  across  all  service 
contracts,  a  semantic  differential  scale  (Table  3)  was  generated  that  broadly  assessed  the 
respondent’s  perception  of  compliance  with  regulations  and  statutes.  The  scale  consists 
of  five  questions  that  were  measured  on  a  7-point  Likert-type  scale  with  anchors  of 
strongly  disagree  and  strongly  agree. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


43 


Table  3.  Construct  Measurement  of  Regulatory  and  Statutory  Compliance 


Scale  item 

Survey  question 

RSCa 

This  contract  is  compliant  with  all  applicable  policy  letters,  the  Federal 
Acquisition  Regulation  and  its  Supplements,  and  procurement  law. 

RSCb 

There  is  nothing  wrong  with  this  contract. 

RSCc 

This  contract  does  not  violate  applicable  requirements  of  policy  letters,  the 
Federal  Acquisition  Regulation  (FAR),  FAR  Supplements,  or  procurement  law. 

RSCd 

If  this  contract  were  to  be  inspected  by  an  independent  organization  it  would  be 
deemed  compliant. 

RSCe* 

There  are  aspects  of  this  contract  that,  if  changed,  would  make  it  more 
compliant  with  policy,  laws,  or  regulations. 

*Reverse-coded  item. 

Similarly,  no  appropriate,  previously  validated  scales  were  available  for  the 
measurement  of  the  sufficiency  of  the  requirement  definition.  Because  contract 
requirements  and,  by  extension,  the  definitions  of  those  requirements  vary  broadly 
between  acquisitions,  the  developed  scale  assessed  the  administrator’s  overall  perception 
of  the  sufficiency  of  the  definition.  The  scale  (Table  4)  consists  of  five  questions 
measured  on  a  7-point  Likert-type  scale  with  anchors  of  strongly  disagree  and  strongly 
agree. 

Table  4.  Construct  Measurement  of  Requirement  Definition  Sufficiency 


Scale  item 

Survey  question 

RDla 

The  requirement  was  very  well  defined  in  the  contract. 

RDlb 

The  contract  (including  the  statement  of  work,  performance  work  statement, 
specification,  drawings,  etc.)  defined  the  requirement  very  well. 

RDlc 

There  were  no  flaws  or  omissions  in  the  definition  of  the  requirement  (including 
the  statement  of  work,  performance  work  statement,  specification,  drawings, 
etc.) 

RDld 

The  requirement,  as  defined  in  the  contract,  expressed  to  the  contractor  exactly 
what  we  needed. 

RDle 

There  were  no  ambiguities  in  the  definition  of  the  requirement  (including  the 
statement  of  work,  performance  work  statement,  specification,  drawings,  etc.) 

JnpIIL 
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The  scale  for  the  measurement  of  communication  (Table  5)  was  adapted  from 
Morgan  and  Hunt  (1994).  In  the  original  three-item  reflective  scale,  all  measures  began 
with  -Til  our  relationship,  my  alliance  partner  and  I  . . .  The  text  was  modified  to  read  as 
follows:  -Tn  our  relationship,  the  government  and  contractor  ...  .”  Additionally,  two 
items  were  added  to  the  scale  to  assess  how  effectively  the  administrator  was  able  to 
communicate  the  government’s  needs  and  how  carefully  the  contractor  listened  to  the 
government’s  requests.  The  final  scale  consists  of  five  questions  measured  on  a  7-point 
Likert-type  scale  with  anchors  of  strongly  disagree  and  strongly  agree. 

Table  5.  Construct  Measurement  of  Communication 

Scale  item  Survey  question 

REa  In  our  relationship,  the  government  and  contractor  effectively  communicate 

expectations  for  each  other’s  performance 

REb  In  our  relationship,  the  government  and  contractor  keep  each  other  informed  of 

new  developments. 

REc  In  our  relationship,  the  government  and  contractor  provide  each  other  with 

information  that  helps  both  parties. 

REd  I  am  able  to  communicate  my  needs  effectively  to  this  contractor. 

REe  This  contractor  listens  carefully  to  my  requests. 


No  appropriate,  previously  validated  scale  was  available  for  the  measurement  of 
the  amount  of  contract  surveillance.  A  scale  was  developed  (Table  6)  based  on 
surveillance  practices  established  in  the  FAR,  in  which  surveillance  occurs  in  four 
primary  areas:  service  quality,  timeliness  of  performance,  fulfillment  of  requirements 
included  in  the  requirements  document,  and  fulfillment  of  other  contract  terms  and 
conditions  (such  as  adherence  to  the  labor  rates  established  by  the  Department  of  Labor). 
The  scale  asked  respondents  to  use  a  7-point  Likert-type  scale  to  rate  the  amount  of 
surveillance  in  each  area.  Scale  anchors  were  no  monitoring  of  supplier  and  extensive 
monitoring  of  supplier. 
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Table  6.  Construct  Measurement  of  Amount  of  Surveillance 


Scale  item 

Survey  question 

ATla 

Service  Quality 

ATlb 

Timeliness  of  Performance 

ATlc 

Fulfillment  of  Requirements  in  the  Statement  of  Work  /  Performance  Work 
Statement 

ATld 

Compliance  with  Contract  Terms  &  Conditions 

The  scale  for  the  measurement  of  internal  customer  commitment  (Table  7)  was 
adapted  from  Garbarino  and  Johnson  (1999),  who  developed  a  four-item  scale  for  the 
measurement  of  customer  commitment.  All  four  items  could  be  generalized  to  fit  a 
business-to-business  context  with  some  modification  and  were  measured  on  a  7-point 
Likert-type  scale  with  anchors  of  no  extent  and  great  extent. 


Table  7.  Construct  Measurement  of  Internal  Customer  Commitment 


Scale  item 

Survey  question 

ICCa 

To  what  extent  is  your  primary  internal  customer  proud  to  be  a  member  of  this 
service  acquisition  team? 

ICCb 

To  what  extent  does  your  primary  internal  customer  feel  a  sense  of  purpose 
when  the  team  achieves  specific  acquisition  objectives? 

ICCc 

To  what  extent  does  your  primary  internal  customer  care  about  the  long-term 
success  of  this  acquisition? 

ICCd 

To  what  extent  is  your  primary  internal  customer  dedicated  to  ensuring  the 
acquisition  meets  requirements? 

No  appropriate,  previously  validated  scales  were  available  for  the  measurement  of 
procurement  lead-time  sufficiency.  A  five-item  scale  was  developed  (Table  8)  to 
measure  the  administrator’s  perception  of  the  adequacy  of  the  procurement  lead-time. 
The  scale  consists  of  five  questions  measured  on  a  7-point  Likert-type  scale  with  anchors 
of  strongly  disagree  and  strongly  agree. 
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Table  8.  Construct  Measurement  of  Procurement  Lead-Time  Sufficiency 


Scale  item 


Survey  question 


RBVla* 

RBVlb* 

RBVlc* 

RBVld 

RBVle 


1  did  not  have  enough  time  to  award  a  quality  contract. 

The  milestones  for  awarding  this  contract  were  too  aggressive. 

My  leadership  or  my  customer  wanted  this  contract  awarded  too  fast. 
1  was  not  rushed  to  award  this  contract. 

1  had  sufficient  time  to  get  this  contract  awarded. 


*Reverse-coded  item. 


Questions  were  constructed  to  measure  single  indicators  and  to  collect 
demographic  data.  First,  7-point  Likert-type  scales  were  employed  to  measure  the  extent 
that  PBSA  procedures  were  used  and  the  extent  that  the  contractor  defined  requirements 
(Table  9).  Anchors  for  both  scales  were  none  and  substantial. 

Table  9.  Measurement  of  Extents  PBSA  and  Contractor  Defined  Requirements 

Variable  Survey  question 

AT2  The  extent  to  which  this  contract  is  performance  based. 

RD2  To  what  extent,  if  any,  did  the  contractor  help  define  the  requirements  prior  to 

contract  award?  (The  contractor's  help  defining  requirements  includes  questions 
before  and  after  the  issuance  of  the  solicitation,  participation  in  industry  days, 
responses  to  requests  for  information,  responses  to  draft  RFQ/RFP,  and  any 
other  dialogue  with  the  contacting  officer  or  specialist.) 


The  number  of  personnel  required  to  administer  a  service  contract  may  vary 
substantially  based  on  scope,  value,  or  other  factors.  Therefore,  personnel  turnover  was 
measured  by  dividing  the  number  of  times  personnel  had  turned  over  by  the  number  of 
personnel  assigned  to  the  contract.  This  allowed  turnover  to  be  measured  as  a  percentage 
and  required  two  questions  for  measurement.  The  survey  questions  for  the  measurement 
of  turnover  can  be  found  in  Table  10. 
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Table  10.  Measurement  of  Turnover 


Variable  Survey  question 

RBV2a  How  many  acquisition  personnel  (contracting  officers  +  contract  specialists  + 
contracting  officer  representatives  +  quality  assurance  evaluators/personnel  + 
inspectors  +  program/project  manager  or  other  active  representative  of  the  user) 
are  currently  assigned  to  this  contract?  Include  only  those  individuals  who 
actively,  routinely  help  manage  the  contract. 

RBV2b  Over  the  life  of  this  contract,  how  many  different  times  have  personnel  turned 
over?  Consider  all  Contract  Administrators,  Contract  Specialists,  Contracting 
Officers,  Contracting  Officer's  Representatives,  and  Quality  Assurance 
Personnel  who  actively,  routinely  helped  manage  the  contract. 


The  contract  administrator’s  levels  of  APDP  certification  and  education  were 
measured  using  ordinal  scales,  and  the  administrator’s  number  of  years  of  contracting 
experience  was  measured  by  text  input  into  a  field.  For  APDP  certification  level, 
possible  responses  were  no  contracting  APDP  certification,  APDP  Level  I,  APDP  Level 
II,  and  APDP  Level  III.  For  level  of  education,  possible  responses  were  high  school 
diploma/GED,  associate’s  degree,  bachelor’s  degree,  master’s  degree,  and 
doctoral/professional  degree. 

Table  11.  Measurement  of  Experience,  APDP  Level,  and  Education  Level 

Variable  Survey  question 

CBV 1  How  many  years  of  experience  do  you  have  in  contracting? 

CBV2  At  the  time  you  started  working  on  this  service  contract,  what  was  the  highest 
level  of  Acquisition  Professional  Development  Program  (APDP)  certification 
that  you  held  in  the  area  of  Contracting? 

CBV3  What  is  the  highest  level  of  education  that  you  have  attained? 


Finally,  questions  were  included  in  the  survey  to  measure  sample  demographics. 
Respondent-related  demographics  included  the  respondent’s  age,  gender,  agency  of 
employment,  workload,  and  whether  the  respondent  held  professional  certifications 
outside  of  those  offered  by  the  APDP.  Demographic  questions  to  assess  the  makeup  of 
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sample  service  contracts  included  the  service  type  and  commerciality;  the  contract  type, 
instrument,  age,  and  value;  the  type  and  amount  of  incentives;  the  type  and  amount  of 
quality-assurance  inspections  and  the  experience  of  inspectors;  the  number  of  formal 
communications  to  the  contractor;  and  a  direct  measure  of  procurement  lead-time.  These 
survey  questions  can  be  found  in  Appendix  A. 

All  survey  questions  were  pre-tested  through  interviews  with  current  U.S.  Air 
Force  contract  administrators  at  three  separate  installations  and  through  reviews  by 
graduate-level  students  and  professors  who  specialize  in  the  field  of  DoD  acquisition. 
Inputs  received  from  these  individuals  on  scale  items  were  used  to  refine  the  measures. 
Interviewees  and  reviewers  were  also  presented  with  a  copy  of  the  conceptual  model  to 
verify  content  validity.  Responses  from  these  individuals  supported  the  proposed 
research  hypotheses  and  did  not  reveal  a  need  to  re-specify  the  conceptual  model  by 
adding  new  relationships  or  by  removing  existing  ones. 

Once  constructed,  the  order  of  the  survey  questions  was  structured  to  reduce  bias 
among  scale  items  by  mixing  questions  with  like  scales;  the  complete  survey  can  be 
found  in  Appendix  A.  As  required  for  the  deployment  of  surveys  for  data  collection  from 
U.S.  Air  Force  personnel,  the  survey  was  reviewed  and  approved  by  the  Air  Force  Survey 
Office.  An  additional  review  was  conducted  through  the  Department  of  the  Navy’s 
Institutional  Review  Board  to  ensure  the  protection  of  human  subjects. 

E.  CONCLUSION 

This  chapter  first  presented  the  research  methodology  and  statistical  method  used 
in  this  study.  A  conceptual  model  was  then  presented,  followed  by  a  discussion  of  the 
population  and  sampling  methods  and  the  survey  design.  The  next  chapter  reports  the 
results  of  the  data  collection  and  statistical  analysis. 
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IV.  RESULTS 


A.  INTRODUCTION 

This  chapter  presents  the  results  of  the  survey  deployment  and  of  the  model  and 
hypothesis  testing.  The  results  of  the  data  collection  efforts  are  discussed  first,  along 
with  a  presentation  of  sample  demographics.  Next,  data  are  examined  to  verify  that  the 
assumptions  of  SEM  analysis  are  met.  These  examinations  include  tests  for  normality, 
outliers,  scale  and  construct  reliability,  convergent  validity,  and  discriminate  validity. 
Exploratory  factor  analysis  (EFA)  is  then  performed  in  PASW  Statistics  Version  18.0  to 
purify  scales,  and  confirmatory  factor  analysis  (CFA)  and  structural  equation  modeling 
(SEM)  are  accomplished  in  Mplus  Version  6.0.  Two  competing  structural  models  are 
presented:  a  hypothesized  structural  model  containing  all  hypothesized  relationships  and 
a  trimmed  structural  model  in  which  non-significant,  originally  hypothesized  causal  paths 
have  been  removed.  Finally,  an  alternative  model  is  analyzed  in  SEM  and  further 
analysis  is  performed  using  other  statistical  methods. 

In  the  presentation  of  the  results  in  this  chapter,  the  level  of  support  for  a  path  was 
determined  by  the  significance  level  of  the  estimated  path  coefficient;  supported  path 
estimates  have  significance  levels  less  than  0.05,  moderately  supported  path  estimates 
have  significance  levels  between  0.05  and  0.10,  and  non-supported  path  estimates  have 
significance  levels  greater  than  0.10.  Path  estimates  and  their  respective  effect  sizes  are 
standardized.  As  opposed  to  unstandardized  estimates,  which  retain  the  scaling 
information  of  those  variables  involved,  the  scaling  of  standardized  estimates  is  not 
linked  to  the  scale  of  the  variables;  they  estimate  the  change  in  standard  deviation  of  the 
dependent  variable  based  on  a  change  of  one  standard  deviation  to  the  independent 
variable  when  all  other  independent  variables  are  held  at  zero  (Hoyle,  1995).  When 
interpreting  standardized  estimates,  a  relationship  with  a  parameter  estimate  greater  than 
0.50  is  considered  to  have  a  large  effect;  a  relationship  with  a  parameter  estimate  between 
0.20  and  0.50  is  considered  to  have  a  moderate  effect;  and  a  relationship  exhibiting  a 
parameter  estimate  less  than  0.20  is  considered  to  have  a  small  effect  (Hair  et  ah,  2010). 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


51 


Because  this  study  used  psychometric  scales  to  measure  unobserved  variables,  it  is  not 
appropriate  to  interpret  the  estimate  as  a  marginal  change;  rather,  it  is  more  appropriate  to 
generalize  the  interpretation  of  estimates  into  effect  sizes  and  to  employ  a  comparative 
approach. 

B.  DATA  COLLECTION 

Data  were  collected  electronically  using  a  web-based  survey  deployment.  Because 
it  is  difficult  to  identify  personnel  who  have  the  necessary  service  contracting  experience 
required  for  response  to  the  survey,  an  invitation  (Appendix  B)  was  forwarded  via  e-mail 
to  eligible  individuals  through  supervisors  of  U.S.  Air  Force  contracting  units.  A 
memorandum  of  support  for  the  study  from  the  Deputy  Assistant  Secretary  of  the  Air 
Force  for  Contracting  and  from  the  Air  Force  Program  Executive  Officer  for  Services 
was  attached  to  the  message  (Appendix  C).  Of  the  60  units  invited  to  participate  in  the 
study,  42  units  agreed  to  participate  and  collectively  distributed  the  survey  to  743 
personnel.  The  only  criterion  given  to  supervisors  for  selecting  personnel  was  that 
respondents  must  either  currently  administer  service  contracts  or  must  have  done  so  in 
the  recent  past.  Of  the  743  potential  participants,  252  individuals  responded,  yielding  a 
34%  response  rate,  which  is  consistent  with  rates  reported  for  web-based  surveys 
(Dillman,  2000).  Within  these  responses,  12  were  later  discarded  due  to  missing,  faulty, 
or  inconsistent  responses.  Summary  statistics  from  the  remaining  240  usable  responses 
can  be  found  in  Table  12. 
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Table  12. 


Variables  (n=240) 

Service  quality 


Regulatory  and 
statutory  compliance 


Requirement  definition 
sufficiency 


Extent  contractor 


Measure 

Mean 

Std.  Dev. 

Min. 

Max. 

SQa 

5.38 

1.34 

2.00 

7.00 

SQb 

5.55 

1.29 

1.00 

7.00 

SQc 

5.77 

1.17 

2.00 

7.00 

SQd 

5.70 

1.18 

3.00 

7.00 

SQe 

5.47 

1.33 

1.00 

7.00 

SQf 

5.62 

1.17 

1.00 

7.00 

SQg 

5.92 

1.13 

2.00 

7.00 

SQh 

5.89 

1.15 

1.00 

7.00 

SQi 

5.49 

1.30 

1.00 

7.00 

SQj 

5.78 

1.23 

1.00 

7.00 

SQk 

5.76 

1.18 

2.00 

7.00 

SQI 

5.79 

1.23 

1.00 

7.00 

SQm 

5.71 

1.25 

1.00 

7.00 

SQn 

5.38 

1.23 

1.00 

7.00 

RSCa 

6.19 

1.16 

1.00 

7.00 

RSCb 

4.78 

1.74 

1.00 

7.00 

RSCc 

6.23 

1.18 

1.00 

7.00 

RSCd 

6.01 

1.10 

1.00 

7.00 

RSCe 

4.80 

1.75 

1.00 

7.00 

RDla 

5.31 

1.49 

1.00 

7.00 

RDlb 

5.25 

1.45 

1.00 

7.00 

RDlc 

4.68 

1.63 

1.00 

7.00 

RDld 

5.53 

1.27 

1.00 

7.00 

RDle 

4.91 

1.59 

1.00 

7.00 

RD2 

2.57 

1.74 

1.00 

7.00 

defined  requirements 


Summary  Statistics 


Variables  (n=240) 

Measure 

Mean 

Std.  Dev. 

Min. 

Max. 

Communication 

REa 

5.73 

1.18 

3.00 

7.00 

REb 

5.65 

1.29 

1.00 

7.00 

REc 

5.70 

1.24 

1.00 

7.00 

REd 

5.88 

1.17 

1.00 

7.00 

REe 

5.80 

1.08 

1.00 

7.00 

Amount  of  surveillance 

ATla 

5.29 

1.49 

1.00 

7.00 

AT  lb 

5.28 

1.52 

1.00 

7.00 

ATlc 

5.34 

1.53 

1.00 

7.00 

ATld 

5.44 

1.48 

1.00 

7.00 

Internal  customer 

ICCa 

5.25 

1.47 

1.00 

7.00 

commitment 

ICCb 

5.40 

1.47 

1.00 

7.00 

ICCc 

5.81 

1.25 

1.00 

7.00 

ICCd 

5.67 

1.33 

1.00 

7.00 

Procurement  lead-time 

RBVla 

4.81 

1.86 

1.00 

7.00 

sufficiency 

RBVlb 

4.59 

1.80 

1.00 

7.00 

RBVlc 

4.45 

1.93 

1.00 

7.00 

RBVld 

4.13 

1.84 

1.00 

7.00 

RBVle 

4.45 

1.69 

1.00 

7.00 

Extent  PBSA 

AT2 

5.60 

1.63 

1.00 

7.00 

Turnover 

RBV2 

1.22 

1.17 

0.00 

7.00 

Buyer  experience 

CBV1 

11.66 

9.88 

0.00 

41.00 

APDP  level 

CBV2 

1.68 

1.06 

0.00 

3.00 

Education  level 

CBV3 

1.98 

0.98 

0.00 

4.00 

c. 


SAMPLE  DEMOGRAPHICS 


From  the  240  usable  responses,  the  average  respondent  was  41.7  years  old  and 
had  11.7  years  of  federal  contracting  experience.  The  gender  of  respondents  was  nearly 
even,  with  males  accounting  for  51.9%  of  respondents  and  females  accounting  for  48.1% 
of  respondents.  Respondents  had  a  diverse  range  of  educational  experience,  with  the 
highest  level  of  education  for  11.9%  of  respondents  being  a  high  school  diploma  or 
general  equivalency  diploma,  the  highest  level  of  education  for  12.3%  of  respondents 
being  an  associate’s  degree,  the  highest  level  of  education  for  43.2%  of  respondents 
being  a  bachelor’s  degree,  and  the  highest  level  of  education  for  31.7%  of  respondents 
being  a  master’s  degree.  Only  two  respondents,  or  0.8%,  held  a  doctoral  or  professional 
degree.  When  the  respondents  were  asked  about  their  Acquisition  Professional 
Development  Program  (APDP)  certification  level  in  the  area  of  contracting,  19.3% 
reported  that  they  held  no  APDP  contracting  certification,  18.5%  reported  that  they  held  a 
Level  I  APDP  contracting  certification,  36.6%  reported  that  they  held  a  Level  II  APDP 
contracting  certification,  and  25.5%  reported  that  they  held  a  Level  III  APDP  contracting 
certification.  Additionally,  37  respondents,  or  15.4%,  reported  that  they  held  a 
professional  certification  other  than  an  APDP  certification,  such  as  those  granted  by  the 
National  Contracting  Management  Association  or  the  Institute  for  Supply  Management. 
Demographics  for  the  education  and  certification  levels  of  the  respondents  can  be  found 
in  Table  13. 
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Table  13.  Education  and  Certification  Demographics 


Education  level 

Percent  of 

total 

High  school  diploma  or  GED 

11.90% 

Associate's  degree 

12.30% 

Bachelor's  degree 

43.20% 

Master's  degree 

31.70% 

Doctoral  or  professional  degree 

00.80% 

Certification  level 

Percent  of 

total 

No  APDP  certification 

19.30% 

APDP  Level  I 

18.50% 

APDP  Level  II 

36.60% 

APDP  Level  III 

25.50% 

Other  professional  certification 

15.40% 

Respondents  were  asked  to  keep  one  specific  service  contract  in  mind  while 
completing  the  survey.  A  diverse  range  of  service  types  were  reported  (Table  14).  Of  the 
respondents,  84.4%  reported  their  contract  type  as  being  fixed-price,  and  9.1%  of 
respondents  reported  their  contract  type  as  being  hybrid,  a  combination  of  multiple 
contract  types.  Of  the  remainder,  1.7%  respondents  reported  labor-hour  contracts,  1.2% 
reported  cost-reimbursement  contracts,  0.8%  reported  time  and  materials  contracts,  and 
2.9%  reported  other.  Additionally,  78.2%  of  respondents  reported  that  the  service  being 
acquired  met  the  definition  of  a  commercial  service,  as  defined  in  the  Federal  Acquisition 
Regulation. 
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Table  14.  Service  Type  Administered  by  Respondents 


Service  Type 

Percent 
of  total 

Maintenance/repair 

22.22 

Professional,  administrative  and  management  support 

17.70 

Utilities  and  housekeeping 

17.28 

Medical 

10.29 

Education  and  training 

4.53 

Architect-engineering 

3.29 

Quality  control,  testing,  and  inspection 

0.82 

Research  and  development 

0.82 

Other 

23.05 

D.  ASSUMPTIONS 

1.  Missing  Data 

Nine  of  the  original  252  responses  were  discarded  due  to  missing  data. 
Three  of  the  remaining  responses  did  not  include  a  properly  reported  age  in  the 
demographic  field;  these  responses  were  not  used  when  reporting  age-related 
demographic  information  but  were  imputed  for  tests  of  non-response  bias  through  mean 
substitution.  An  additional  five  responses  included  a  plus  symbol  as  a  qualifier  in  a  total 
of  six  data  fields  and  two  responses  included  the  preposition  over  in  a  total  of  two  data 
fields.  In  each  of  these  cases,  responses  were  assumed  to  be  1  plus  the  numerical 
response. 
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2. 


Outliers 


Prior  to  performing  tests  for  the  detection  of  outliers,  data  were  first 
assessed  for  faulty  or  inconsistent  responses.  The  inclusion  of  four  reverse-coded  survey 
items  (Churchill,  1979)  allowed  for  the  identification  of  firewalled  responses  (identical 
responses  for  all  survey  questions),  where  a  firewalled  response  would  likely  also  result 
in  the  identification  of  a  case  as  a  multivariate  outlier.  A  total  of  three  survey  responses 
were  identified  as  faulty  due  to  an  examination  of  the  reverse-coded  items  and  an 
assessment  of  the  overall  survey  response,  including  any  comments  left  by  the  respondent 
in  the  comment  field.  Each  of  these  responses  was  discarded. 

Next,  the  presence  of  univariate  and  multivariate  outliers  was  assessed 
using  PASW  Statistics  Version  18.0.  To  detect  univariate  outliers,  responses  for  each 
variable  were  saved  as  standardized  values  and  evaluated  against  a  benchmark  in  which 
potential  outliers  had  z-scores  outside  of  ±3.00.  Each  potential  case  was  reviewed 
individually;  none  appeared  to  be  invalid  or  warranted  removal  from  the  data  set. 

An  assessment  for  the  presence  of  multivariate  outliers  was  performed  by 
computing  Mahalanobis  Distance  using  a  multiple  regression  in  PASW  Statistic  Version 
18.0  in  which  the  dependent  variable  was  arbitrarily  selected  and  all  other  variables  were 
included  as  independent  variables  (Table  14);  associated  probabilities  were  calculated 
using  the  chi-square  distribution.  Byme  (2010)  asserted  that  a  case  containing 
multivariate  outliers  is  identifiable  when  the  Mahalanobis  D"  stands  distinctly  apart  from 

2  9 

other  D  values.  A  review  of  D  values  in  Table  15,  which  contains  the  highest  60 
values,  reveals  a  distribution  in  values  from  61  to  142,  with  12  cases  exceeding  a  D“ 
value  of  100.  All  cases  with  a  probability  less  than  or  equal  to  0.001  were  individually 
reviewed;  none  appeared  to  be  invalid  or  warranted  removal  from  the  data  set. 
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Table  15.  Outlier  Assessment 


Record 

Mahalanobis 

D-Squared 

P- value 

Record 

Mahalanobis 

D-Squared 

P-value 

Record 

Mahalanobis 

D-Squared 

P-value 

146 

142.18 

<  0.001 

182 

78.86 

0.003 

118 

69.24 

0.024 

96 

133.28 

<  0.001 

15 

78.53 

0.004 

6 

68.00 

0.030 

82 

124.06 

<  0.001 

117 

78.31 

0.004 

73 

67.08 

0.036 

9 

120.29 

<  0.001 

4 

77.98 

0.004 

112 

66.28 

0.041 

125 

111.63 

<  0.001 

84 

77.00 

0.005 

76 

66.27 

0.041 

54 

109.94 

<  0.001 

56 

76.77 

0.005 

91 

66.21 

0.042 

143 

109.40 

<  0.001 

3 

75.92 

0.006 

128 

65.94 

0.044 

217 

107.04 

<  0.001 

113 

75.45 

0.007 

225 

65.26 

0.049 

115 

104.58 

<  0.001 

150 

75.35 

0.007 

162 

64.75 

0.054 

199 

102.15 

<  0.001 

62 

74.69 

0.008 

20 

64.52 

0.056 

122 

100.65 

<  0.001 

58 

73.07 

0.011 

98 

64.50 

0.056 

104 

100.25 

<  0.001 

22 

72.69 

0.012 

52 

64.49 

0.056 

169 

97.96 

<  0.001 

216 

72.19 

0.014 

179 

64.27 

0.058 

34 

94.61 

<  0.001 

207 

71.17 

0.017 

23 

63.73 

0.064 

31 

88.78 

<  0.001 

39 

70.95 

0.017 

200 

63.00 

0.072 

46 

88.53 

<  0.001 

221 

70.71 

0.018 

141 

62.28 

0.081 

126 

87.89 

<  0.001 

194 

70.59 

0.019 

108 

62.09 

0.083 

173 

85.35 

0.001 

53 

69.92 

0.021 

119 

61.96 

0.085 

190 

82.93 

0.001 

65 

69.61 

0.022 

237 

61.34 

0.094 

185 

82.89 

0.001 

154 

69.38 

0.023 

61 

61.27 

0.095 

3.  Normality 


An  assumption  of  normality  is  extremely  important  to  the  analysis  of 
structural  equation  models  (Arbuckle,  2007).  The  maximum  likelihood  method  of 
estimation,  the  most  commonly  used  method  of  estimation  in  SEM  and  the  method  that 
was  used  in  this  study,  is  considered  to  be  robust  against  modest  levels  of  non-normality 
(Chou  &  Bentler,  1995).  Alternative  methods  of  estimation,  such  as  asymptotically 
distribution-free  or  generalized  least  squares,  may  be  used  when  data  depart  substantially 
from  nonnality,  but  these  methods  typically  require  samples  of  substantial  size  (Muthen 
&  Kaplan,  1992). 

Although  normally  distributed  data  take  the  form  of  a  symmetrical  bell 
curve,  non-normally  distributed  data  may  exhibit  a  curve  that  is  skewed  in  either 
direction  or  that  is  abnormally  peaked,  termed  kurtosis.  A  distribution  with  too  few  cases 
in  the  tails  of  the  curve  (positive  kurtosis)  will  exhibit  a  higher-than-normal  peak. 
Alternatively,  a  distribution  with  too  many  cases  in  the  tails  of  the  curve  (negative 
kurtosis)  will  exhibit  a  flatter-than-normal  peak  (DeCarlo,  1997). 

Table  16  displays  an  assessment  of  normality  that  was  produced  using 
descriptive  statistics  in  PASW  Statistics  Version  18.0.  Using  the  benchmark  of  ±  2.0  for 
significant  skewness  (West,  Finch,  &  Curran,  1995),  a  review  of  the  values  in  the  table 
does  not  reveal  that  distributions  for  any  of  the  variables  depart  substantially  from 
normality  as  a  result  of  skew. 

The  assessment  of  normality  also  displays  test  statistics  for  kurtosis.  In 
the  analysis  of  kurtosis,  normal  distributions  display  a  standardized  kurtosis  value  ((U  )  of 
3.0,  and  rescaled  distributions  display  a  (32  value  of  0.  West,  Finch,  and  Curran  (1995)  - 
suggested  that  rescaled  \h  values  greater  than  or  equal  to  seven  are  indicative,  to  some 
extent,  of  non-normality.  Since  none  of  the  (f  values  in  the  assessment  exceed  this 
threshold,  it  can  be  assumed  that  univariate  kurtosis  does  not  exist  and  that  the 
assumption  of  normality  is  not  violated. 
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Table  16.  Normality  Assessment 


Variable 

Min. 

Max. 

Skew 

Cricital 

Region 

Kurtosis 

(Pi) 

Cricital 

Region 

Variable 

Min. 

Max. 

Skew 

Cricital 

Region 

Kurtosis 

(P2) 

Cricital 

Region 

SQa 

2.00 

7.00 

-0.53 

-3.35 

-0.64 

-2.01 

REa 

3.00 

7.00 

-0.74 

-4.69 

-0.33 

-1.05 

SQb 

1.00 

7.00 

-0.81 

-5.12 

0.37 

1.16 

REb 

1.00 

7.00 

-1.00 

-6.31 

0.71 

2.25 

SQc 

2.00 

7.00 

-0.78 

-4.95 

-0.18 

-0.55 

REc 

1.00 

7.00 

-0.92 

-5.81 

0.72 

2.29 

SQd 

3.00 

7.00 

-0.64 

-4.05 

-0.59 

-1.88 

REd 

1.00 

7.00 

-1.66 

-10.51 

3.77 

11.91 

SQe 

1.00 

7.00 

-0.70 

-4.40 

-0.07 

-0.23 

REe 

1.00 

7.00 

-0.96 

-6.06 

1.23 

3.89 

SQf 

SQg 

SQh 

SQi 

SQj 

1.00 

2.00 

1.00 

1.00 

1.00 

7.00 

7.00 

7.00 

7.00 

7.00 

-0.80 

-0.97 

-0.99 

-0.84 

-1.01 

-5.04 

-6.10 

-6.25 

-5.32 

-6.39 

0.72 

0.50 

0.76 

0.62 

0.62 

2.28 

1.58 

2.39 

1.97 

1.97 

ATla 

ATlb 

ATlc 

ATld 

1.00 

1.00 

1.00 

1.00 

7.00 

7.00 

7.00 

7.00 

-0.87 

-0.87 

-0.88 

-0.93 

-5.49 

-5.47 

-5.57 

-5.87 

0.36 

0.25 

0.23 

0.44 

1.15 

0.79 

0.73 

1.40 

SQk 

2.00 

7.00 

-0.77 

-4.86 

-0.09 

-0.27 

ICCa 

1.00 

7.00 

-0.73 

-4.60 

0.01 

0.03 

SQI 

1.00 

7.00 

-1.03 

-6.52 

0.82 

2.59 

ICCb 

1.00 

7.00 

-0.81 

-5.11 

0.39 

1.22 

SQm 

1.00 

7.00 

-0.93 

-5.85 

0.43 

1.37 

ICCc 

1.00 

7.00 

-1.04 

-6.58 

0.78 

2.45 

SQn 

1.00 

7.00 

-0.47 

-2.95 

-0.08 

-0.26 

ICCd 

1.00 

7.00 

-0.89 

-5.62 

0.10 

0.31 

RSCa 

1.00 

7.00 

-1.82 

-11.48 

3.75 

11.85 

RBVla 

1.00 

7.00 

-0.48 

-3.01 

-0.89 

-2.81 

RSCb 

1.00 

7.00 

-0.62 

-3.91 

-0.54 

-1.72 

RBVlb 

1.00 

7.00 

-0.32 

-1.99 

-0.84 

-2.67 

RSCc 

1.00 

7.00 

-1.95 

-12.36 

4.36 

13.79 

RBVlc 

1.00 

7.00 

-0.34 

-2.16 

-0.96 

-3.04 

RSCd 

1.00 

7.00 

-1.41 

-8.89 

2.41 

7.63 

RBVld 

1.00 

7.00 

-0.03 

-0.16 

-0.94 

-2.98 

RSCe 

1.00 

7.00 

-0.38 

-2.43 

-0.93 

-2.94 

RBVle 

1.00 

7.00 

-0.28 

-1.80 

-0.54 

-1.72 

RDla 

1.00 

7.00 

-0.91 

-5.76 

0.34 

1.06 

AT2 

1.00 

7.00 

-1.28 

-8.10 

1.08 

3.42 

RDlb 

RDlc 

1.00 

1.00 

7.00 

7.00 

-0.78 

-0.42 

-4.91 

-2.68 

0.07 

-0.65 

0.22 

-2.05 

RBV2 

0.00 

7.00 

1.93 

12.18 

4.94 

15.61 

RDld 

1.00 

7.00 

-0.99 

-6.27 

0.83 

2.61 

CBV1 

0.00 

41.00 

0.83 

5.22 

-0.37 

-1.18 

RDle 

1.00 

7.00 

-0.67 

-4.24 

-0.21 

-0.67 

CBV2 

0.00 

3.00 

-0.33 

-2.07 

-1.10 

-3.48 

RD2 

1.00 

7.00 

0.85 

5.36 

-0.34 

-1.07 

CBV3 

0.00 

4.00 

-0.68 

-4.32 

-0.31 

-0.97 

4. 


Non-Response  Bias 


Non-response  bias  was  estimated  using  extrapolation  methods  (Armstrong 
&  Overton,  1977).  The  extrapolation  method  is  a  commonly  used  method  to  estimate 
bias  when  responses  are  received  in  successive  waves  after  follow-up  communications  to 
non-respondents  (Filion,  1976).  Responses  in  this  study  were  grouped  into  waves 
according  to  the  order  of  arrival,  with  non-respondents  belonging  to  an  unobserved  wave 
that  most  closely  resembles  the  final  wave  of  respondents  (Armstrong  &  Overton,  1977). 
Since  non-responders  were  least  likely  to  resemble  early  responders  and  were  most  likely 
to  resemble  late  responders,  differences  between  early  and  late  responders  were  explored 
to  detect  bias  using  tests  of  equality  of  means.  When  using  extrapolation  methods,  if 
these  tests  do  not  produce  sufficient  statistical  evidence  to  reject  the  null  hypotheses  that 
the  means  are  equal,  then  it  is  not  unreasonable  to  assert  that  the  unobserved  group  of 
non-respondents  has  similar  characteristics  to  those  of  the  survey  respondents  and  that 
non-respondents  are  missing  at  random. 

As  such,  responses  were  categorized  into  three  groups.  Group  1 
represented  the  first  third  of  respondents  to  complete  the  survey;  Group  2  represented  the 
second  third  of  respondents;  and  Group  3  represented  the  final  third  of  respondents  (i.e., 
the  latest  responders).  A  multivariate  analysis  of  variance  (MANOVA)  was  then 
conducted  against  age,  years  of  contracting  experience,  and  three  latent  variables  in  order 
to  explore  differences.  The  three  latent  variables — requirement  definition  sufficiency, 
regulatory  and  statutory  compliance,  and  service  quality — were  computed  using 
summations  of  scale  items.  Additionally,  two  categorical  variables,  gender  and  education 
level,  were  examined  for  differences  in  means  between  the  waves  using  Pearson’s  chi- 
square  test.  Individual  hypotheses  are  located  in  Table  17  and  descriptive  statistics  for 
the  variables  tested  can  be  found  in  Appendix  D. 
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Table  17.  Non-Response  Bias  Hypotheses 


Variable 

Hypothesis 

Significance 

Age 

Ho-  [^Agel  [^Age2  |^Age3 

a  =  0.05 

Gender 

Ho-  [^Genderl  ^Gender2  |^Gender3 

a  =  0.05 

Buyer  experience 

Ho-  [^Experience  1  |^Experience2  [-^Experiences 

a  =  0.05 

Education  level 

Ho*  [^Educationl  [^Education2  [^Education3 

a  =  0.05 

Service  quality 

Ho*  H-Qualityl  [^Quality2  |^Quality3 

a  =  0.05 

Requirement  definition 

Ho*  [^Definition  1  [^Definition2  [^Definitions 

a  =  0.05 

Regulatory  and 

H0.  [ACompliancel  [^Compliance2  [-^Compliances 

a  =  0.05 

statutory  compliance 

Homogeneity  of  variances  is  a  critical  underlying  assumption  to  the 
analysis  of  variance  (Kerlinger  &  Lee,  2000).  If  variances  between  groups  are  not  equal, 

or  homoscedastic,  then  correlation  coefficients  may  reflect  error.  The  following  was 

2  2  2 

hypothesized:  given  each  variable  X,  £  XI  -  C’X2  -  C  X3-  Levene’s  test  was  applied  to 
each  variable  and  test  statistics  were  calculated.  Reviewing  the  results  in  Table  18,  no 
statistic  was  significant  at  a  =  0.05,  and  none  of  the  null  hypotheses  can  be  rejected. 
There  is  little  evidence  that  the  variances  are  not  equal,  and  homogeneity  of  variances  can 
be  assumed  for  all  seven  variables. 

A  multivariate  analysis  of  variance  was  conducted  in  PASW  Statistics 
Version  18.0  to  test  the  first  five  of  the  seven  hypotheses;  gender  and  education  level 
were  later  tested.  F  tests  for  Pillai’s  Trace,  Wilks’  Lambda,  Hotelling’s  Trace,  and  Roy’s 
Largest  Root  were  performed;  no  F  test  statistics  were  significant.  Additional  results  of 
the  univariate  F  tests  are  presented  in  Table  19.  An  examination  of  the  significance  of 
the  F  test  statistics  reveals  that  none  are  significant  at  a  level  of  0.05.  Therefore,  there  is 
insufficient  statistical  evidence  to  reject  the  null  hypothesis  that  group  means  are  equal  in 
any  of  the  five  variables,  and  it  can  be  assumed  that  no  bias  exists  between  survey 
responders  and  survey  non-responders  in  these  cases  (Armstrong  &  Overton,  1977). 
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Table  18.  Homogeneity  of  Variance 


Variable 

Levene 

statistic 

dfl 

df2 

Sig. 

Age 

0.727 

2 

237 

0.485 

Gender 

0.298 

2 

237 

0.742 

Buyer  experience 

0.067 

2 

237 

0.936 

Education  level 

1.386 

2 

237 

0.252 

Sufficient  requirement  definition 

0.230 

2 

237 

0.795 

Regulatory  and  statutory  compliance 

0.651 

2 

237 

0.523 

Service  quality 

0.117 

2 

237 

0.890 

Table  19.  Univariate  F  Tests  for  Non-Response  Bias 


Variable 

Sum  of 

squares 

df 

Mean 

square 

F 

Sig. 

Age 

Between  groups 

259.65 

2 

129.83 

0.88 

0.41 

Within  groups 

34804.67 

237 

146.86 

Total 

35064.32 

239 

Buyer 

Between  groups 

183.66 

2 

91.83 

0.93 

0.40 

experience 

Within  groups 

23425.38 

237 

98.84 

Total 

23609.04 

239 

Sufficient 

Between  groups 

14.70 

2 

7.35 

0.19 

0.83 

requirement 

Within  groups 

9261.76 

237 

39.08 

definition 

Total 

9276.46 

239 

Regulatory  and 

Between  groups 

24.93 

2 

12.46 

0.47 

0.63 

statutory 

Within  groups 

6264.93 

237 

26.43 

compliance 

Total 

6289.85 

239 

Service  quality 

Between  groups 

331.86 

2 

165.93 

0.78 

0.46 

Within  groups 

50378.64 

237 

212.57 

Total 

50710.50 

239 
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The  final  two  variables,  gender  and  education  level,  were  tested  for  non¬ 
response  bias  using  Pearson’s  chi-square  test.  Crosstabs  and  test  results  for  these  two 
variables  are  located  in  Table  20  and  Table  21.  As  seen  in  the  tables,  neither  test  was 
significant  at  an  alpha  level  of  0.05;  the  null  hypotheses  stating  that  group  means  are 
equal  cannot  be  rejected  for  either  of  the  two  variables. 


Table  20.  Chi-Square  Test  of  Gender  for  Non-Response  Bias 


Female 

Male 

Total 

1 

41 

39 

80 

Response  Wave 

2 

35 

45 

80 

3 

39 

41 

80 

Total 

115 

125 

240 

Value 

df 

Asymp.  Sig. 
(2-sided) 

Pearson  Chi-Square 

0.935 

2 

0.627 

Likelihood  Ratio 

0.936 

2 

0.626 

N  of  Valid  Cases 

240 

Table  21.  Chi-Square  Test  of  Education  Level  for  Non-Response  Bias 


High 

School 

Associate's 

Degree 

Bachelor's 

Degree 

Master's 

Degree 

PhD  /  Pro 

Degree 

Total 

Response 

1 

7 

11 

37 

25 

0 

80 

Wave 

2 

13 

8 

30 

28 

1 

80 

3 

9 

9 

37 

24 

1 

80 

Total 

29 

28 

104 

77 

2 

240 

Value 

df 

Asymp.  Sig.  (2-sided) 

Pearson  Chi-Square 

4.711 

8 

0.788 

Likelihood  Ratio 

5.313 

8 

0.724 

N  of  Valid  Cases 

240 
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In  summary,  seven  variables  were  tested  for  potential  effects  due  to  the 
presence  of  non-response  bias  using  multiple  methods.  The  results  of  testing  did  not 
provide  statistical  evidence  to  reject  the  null  hypotheses  stating  that  the  means  of 
variables  are  equal  between  the  first  third,  middle  third,  and  final  third  of  respondents. 
As  such,  it  can  be  assumed  that  non-respondents  are  missing  from  the  sample  at  random 
and  that  further  statistical  analysis  will  not  yield  biased  estimates  due  to  non-response. 

5.  Sample  Size 

The  analysis  of  structural  equations  is  often  considered  a  large-sample 
technique  (Ullman,  2006).  Although  the  maximum  likelihood  method  of  estimation  can 
provide  stable  and  valid  results  under  ideal  conditions  with  sample  sizes  as  low  as  50, 
parameter  estimates  and  the  chi-square  test  of  fit  are  known  to  be  sensitive  to  the  size  of 
the  sample  (Hair  et  ah,  2010).  As  a  result,  researchers  have  developed  several  rules  of 
thumb  regarding  adequate  sample  sizes,  in  which  many  have  advocated  for  a  minimum 
sample  size  of  200  and  others  have  recommended  minimum  sample  sizes  as  low  as  100 
(Smith  &  Langfield-Smith,  2004).  Hair  et  al.  (2010)  considered  model  complexity  when 
recommending  a  minimum  sample  size,  and  they  posited  that  a  minimum  of  150  samples 
is  generally  adequate  for  models  with  seven  or  fewer  constructs,  modest  communalities, 
and  no  underdefined  constructs;  such  is  the  case  with  this  study.  Because  240  responses 
are  available  for  analysis,  the  sample  size  for  this  study  exceeded  the  minimum 
recommended  by  most  authors  and  was  appropriately  large  when  model  complexity  and 
the  method  of  estimation  are  considered. 

E.  EXPLORATORY  FACTOR  ANALYSIS 

As  a  preliminary  step  to  establishing  construct  validity,  scales  were  purified  using 
exploratory  factor  analysis  in  PASW  Statistics  Version  18.0.  Principle  components 
factor  analysis  was  first  perfonned  on  a  group  consisting  of  all  exogenous  structures  and 
then  separately  performed  on  a  group  consisting  of  all  endogenous  structures.  Items  were 
discarded  if  they  presented  low  component-factor  scores,  significant  cross-loadings  on 
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other  factors,  or  were  detrimental  to  construct  reliability.  Combined,  42  items  were 
reduced  to  27  across  seven  constructs. 

As  expected,  items  for  the  four  exogenous  constructs  extracted  into  four  factors 
using  Varimax  rotation  when  minimum  eigenvalues  for  factors  were  set  at  value  of  1 
(Table  22).  Initial  eigenvalues  showed  that  the  first  factor  explained  37%  of  the  variance, 
the  second  factor  20%  of  the  variance,  the  third  factor  12%  of  the  variance,  and  the  final 
factor  10%  of  the  variance.  In  total,  the  four  factors  explained  79%  of  the  variance;  a 
scree  plot  of  eigenvalues  by  factor  is  located  in  Appendix  E.  All  but  one  item  displayed 
cross-loadings  below  the  0.300  limit  recommended  by  Hair  et  al.  (2010).  This  item,  a 
measurement  item  from  the  communication  scale,  presented  a  cross-loading  of  0.324  on 
the  factor  consisting  of  scale  items  for  internal  customer  commitment;  however,  the  item 
had  a  strong  primary  loading  of  0.776  on  its  own  factor  and  was  therefore  retained.  All 
construct  items  loaded  cleanly  on  their  respective  factors  with  factor  loadings  above  the 
0.400  benchmark.  Descriptive  statistics  for  the  resultant  factors  are  located  in  Table  23. 


Table  22.  Exogenous  Construct  Component  Matrix 


Item 

1 

Factor 

2  3 

4 

ATla 

0.922 

0.121 

0.000 

0.046 

ATlc 

0.934 

0.145 

0.030 

0.100 

ATld 

0.946 

0.049 

0.034 

0.111 

ICCa 

0.134 

0.891 

0.121 

0.191 

ICCb 

0.093 

0.898 

0.158 

0.216 

ICCd 

0.109 

0.814 

0.218 

0.153 

RBVlc 

-0.015 

0.111 

0.766 

0.122 

RBVld 

0.003 

0.186 

0.850 

0.145 

RBVle 

0.075 

0.141 

0.871 

0.093 

REa 

0.112 

0.290 

0.205 

0.749 

REb 

0.032 

0.324 

0.090 

0.776 

REd 

0.109 

0.014 

0.105 

0.804 
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Table  23.  Descriptive  Statistics  for  Exogenous  Constructs 


Summated  construct 

Items 

Min. 

Max. 

M  (SD) 

Skewness 

Kurtosis 

Amount  of  surveillance 
(ATI) 

3 

3 

21 

16.08  (4.25) 

-0.90 

0.49 

Internal  customer 
Commitment  (ICC) 

3 

5 

21 

16.32  (3.81) 

-0.69 

-0.05 

Procurement  lead-time 
sufficiency  (RBV1) 

3 

3 

21 

13.03  (4.63) 

-0.21 

-0.35 

Communication  (RE) 

3 

8 

21 

17.27  (2.99) 

-0.70 

-0.09 

Items  for  the  three  endogenous  constructs  extracted  cleanly  into  three  factors 
(Table  25).  As  before,  Varimax  rotation  was  selected  and  a  minimum  value  of  1  was 
established  for  factor  eigenvalues.  Initial  eigenvalues  showed  that  the  first  factor 
explained  51%  of  the  variance,  the  second  factor  16%  of  the  variance,  and  the  third  factor 
10%  of  the  variance.  In  total,  these  three  factors  explained  76%  of  the  variance;  a  scree 
plot  of  eigenvalues  by  factor  is  located  in  Appendix  F.  As  anticipated,  scale  items  for  all 
three  service  quality  dimensions  presented  strong  loadings  on  the  same  factor,  indicating 
that  these  dimensions  share  a  primary  factor  and  evidencing  construct  validity.  No  scale 
items  from  any  constructs  displayed  cross-loadings  above  the  0.300  recommended 
threshold  and  all  construct  items  presented  factor  loadings  greater  than  0.400  on  their 
respective  factors.  Descriptive  statistics  for  the  resultant  factors  are  located  in  Table  24. 


Table  24.  Descriptive  Statistics  for  Endogenous  Constructs 


Summated  construct 

Items 

Min. 

Max. 

M  (SD) 

Skewness 

Kurtosis 

Service  quality  (SQ) 

8 

21 

56 

45.21  (8.68) 

-0.65 

-0.43 

Regulatory  and  statutory 
compliance  (RSC) 

3 

4 

21 

17.22  (3.30) 

-1.01 

1.13 

Requirement  definition 
sufficiency  (RD1) 

3 

3 

21 

15 .47  (4.04) 

-0.63 

-0.07 
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Table  25.  Endogenous  Construct  Component  Matrix 


Item 

1 

Factor 

2 

3 

SQa 

0.804 

0.265 

0.055 

SQc 

0.921 

0.140 

0.079 

SQd 

0.904 

0.142 

0.098 

SQf 

0.763 

0.113 

0.081 

SQi 

0.826 

0.174 

0.093 

SQj 

0.901 

0.148 

0.105 

SQk 

0.898 

0.184 

0.071 

SQm 

0.894 

0.183 

0.078 

RSCa 

0.203 

0.209 

0.853 

RSCc 

0.155 

0.115 

0.878 

RSCe 

-0.038 

0.093 

0.658 

RDla 

0.231 

0.866 

0.138 

RDlb 

0.220 

0.880 

0.153 

RDle 

0.180 

0.811 

0.176 

Finally,  scale  reliability  was  assessed  using  PASW  Statistics  Version  18.0. 
Cronbach’s  alpha  is  the  most  widely  used  measure  of  scale  reliability  and  consistency 
(Flair  et  ah,  2010),  in  which  reliability  coefficients  range  from  0.00  to  1.00  and  scales 
with  coefficients  greater  than  0.70  are  generally  considered  to  be  reliable  (Nunnally, 
1978).  As  seen  in  Table  26,  reliability  coefficients  for  the  seven  constructs  ranged  from 
0.70  to  0.96,  with  all  scales  exceeding  the  recommended  minimum. 
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Table  26.  Endogenous  and  Exogenous  Construct  Reliability 


Latent  structure 

Title 

Cronbach’s  alpha 

hi 

Service  quality 

0.959 

h2 

Regulatory  and  statutory  compliance 

0.702 

h3 

Sufficient  requirement  definition 

0.870 

$1 

Communication 

0.757 

^2 

Amount  of  surveillance 

0.939 

^3 

Sufficient  procurement  lead-time 

0.806 

^4 

Internal  customer  commitment 

0.901 

F.  CONFIRMATORY  FACTOR  ANAFYSIS 

1.  Introduction 

Confirmatory  factor  analysis  (CFA)  is  a  method  of  testing  how  well 
manifest  variables  represent  their  respective  constructs.  CFA  differs  from  EFA  in  that 
CFA  is  a  confirmatory  test  of  a  priori  measurement  theory,  and  EFA  identifies  factors 
solely  on  statistical  results  (Hair  et  ah,  2010).  In  this  respect,  a  primary  objective  of  CFA 
is  to  assess  construct  validity  and  fit  of  measurement  models  prior  to  perfonning  further 
analysis  through  the  use  of  structural  equation  modeling  or  other  statistical  techniques. 

Measurement  models  are  CFA  models  that  test  measurement  assumptions 
of  latent  variables,  such  as  construct  validity  (Bagozzi,  1983),  with  no  constraints  on 
correlations  between  constructs.  Mulaik  and  James  (1995)  asserted  that  researchers 
should  not  proceed  to  test  a  structural  model  if  a  measurement  model  exhibits  poor  fit. 
Generally,  the  fit  of  a  model  is  the  extent  that  the  implied  covariance  matrix  is  equivalent 
to  the  observed  population  covariance  matrix  from  which  the  sample  is  drawn  (Hoyle, 
1995).  While  the  most  common  assessment  of  this  equivalence  is  through  the  use  of  the 
chi-square  test  statistic,  this  test  is  known  to  penalize  large  samples  and  complex  models; 
as  a  result,  researchers  have  developed  and  adopted  alternative  measures  of  fit  (Hu  & 
Bentler,  1995).  These  measures  of  fit  are  often  categorized  into  three  groups:  absolute  fit 
indices  (of  which  chi-square  is  a  member),  incremental  fit  indices,  and  parsimony  fit 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


69 


indices.  Because  no  generally  accepted  guideline  exists  as  to  which  fit  indices 
researchers  should  report,  multiple  indices  are  often  used  with  one  or  more  fit  indices 
coming  from  each  category.  The  fit  indices  reported  in  this  study  confonn  to  this 
prescription  but  are  limited  to  those  indices  that  are  produced  by  the  SEM  software 
Mplus  Version  6.0:  chi-square  (x  );  relative  chi-square  (x  / df);  the  Comparative  Fit  Index 
(CF1);  the  Tucker-Lewis  Index  (TL1),  which  is  also  known  as  the  Non-normed  Fit  Index 
(NNFI);  the  Root  Mean  Square  Error  of  Approximation  (RMSEA);  and  the  Standardized 
Root  Mean  Square  Residual  (SRMR). 

Hair  et  al.  (2010)  offered  four  rules  of  thumb  for  assessing  construct 
validity.  First,  these  authors  recommended  that  standardized  loading  estimates  should, 
ideally,  be  higher  than  0.70  but  generally  higher  than  0.50.  Second,  for  convergent 
validity,  average  variance  extracted  (AVE)  for  latent  constructs  should  be  at  least  0.50, 
meaning  that  at  least  50%  of  variance  is  explained  among  the  items  of  a  construct.  Third, 
for  discriminant  validity,  AVE  for  any  given  construct  should  be  higher  than  the  square 
of  the  correlation  between  the  construct  and  any  other  construct.  Finally,  for  internal 
consistency,  construct  reliability  coefficients  should  be  0.70  or  higher.  Because  construct 
reliabilities  were  previously  found  to  be  adequate,  with  all  Cronbach’s  alpha  coefficients 
exceeding  the  guideline  of  0.70,  the  three  remaining  elements,  or  rules  of  thumb,  for 
assessing  construct  validity  will  be  examined  throughout  this  section. 

For  uniformity  with  EFA  methods,  CFA  was  performed  separately  on 
exogenous  and  endogenous  constructs.  However,  CFA  was  first  performed  on  a 
combined  measurement  model  containing  all  exogenous  and  endogenous  constructs 
(Appendix  G)  to  assess  nomological  validity.  Pearson  correlation  coefficients  as  well  as 
directionality  of  correlations  between  constructs  (endogenous  or  exogenous)  and 
endogenous  constructs  were  examined.  Of  these  correlations,  all  were  in  the  expected 
direction  and  were  significant  with  a  p  value  of  less  than  0.01,  indicating  acceptable 
evidence  of  nomological  validity  (Hair  et  al.,  2010). 
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2.  Exogenous  Measurement  Model 

Exogenous  structures  were  first  examined  for  model  fit  and  construct 
validity  using  confirmatory  factor  analysis  with  maximum  likelihood  estimation  (MLE) 
in  Mplus  Version  6.0.  In  the  path  diagram  of  the  measurement  model  (Figure  7),  latent 


Fit  of  the  measurement  model  (Table  27)  was  initially  examined  using  the 
chi-square  test  statistic;  the  chi-square  (%  )  value  of  52.72  was  not  significant  with  a  p 
value  of  0.30,  suggesting  good  fit.  The  relative  chi-square  value  (y  divided  by  degrees 


of  freedom)  of  1.10  also  indicates  good  fit  when  assessed  against  the  threshold  of  3.0 
recommended  by  Kline  (1998).  A  global  assessment  of  model  lit  was  also  accomplished 
using  the  other  fit  indices  produced  by  Mplus  Version  6.0.  The  CFI  value  of  1.00  and 
TLI  value  of  1.00  are  both  higher  than  the  0.95  minimums  that  were  proposed  by  Hair  et 
al.  (2010),  indicating  good  lit.  Similarly,  the  RMSEA  value  of  0.02  and  SRMR  value  of 
0.03  indicate  good  lit  and  are  less  than  the  thresholds  proposed  by  Hair  et  al.  (2010),  who 
recommended  a  maximum  value  of  0.08  for  each.  Overall,  this  assessment  of  fit 
indicates  that  the  exogenous  measurement  model  appears  to  lit  well  to  the  sample  data. 


Table  27.  Exogenous  Measurement  Model  Diagnostics 


X2/df 

CFI 

TLI 

RMSEA 

SRMR 

Measurement  model 

1.10 

1.00 

1.00 

0.02 

0.03 

Fitness  criterion 

<3.00 

>0.95 

>0.95 

<0.08 

<0.08 

An  examination  of  factor  loadings  from  the  results  of  the  confirmatory 
factor  analysis  (Table  28)  revealed  all  standardized  loadings  to  be  above  0.50,  the 
minimum  loading  recommended  by  Hair  et  al.  (2010).  Nearly  all  factor  loadings  from 
the  exogenous  measurement  model  also  meet  the  authors’  more  rigorous  guideline  of 
0.70,  which  they  considered  to  be  ideal.  Furthermore,  all  constructs  exceed  the  minimum 
50%  AVE  also  recommended  by  Hair  et  al.  (2010),  where  AVE  for  each  construct  was 
calculated  by  summating  the  squared  factor  loadings  for  construct  items  and  dividing  the 
summation  by  the  number  items  measuring  the  construct: 

n 

YA 


(1) 


AVE 


i- 1 


n 


Finally,  the  results  of  the  estimation  were  examined  for  the  presence  of 
Heywood  Cases,  or  instances  of  negative  measurement  error  variance.  A  review  of  error 
variance,  which  can  be  found  in  Appendix  I,  does  not  reveal  any  such  cases. 
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Table  28.  Exogenous  Construct  Reliability  and  Convergent  Validity 


Construct 

Items 

Standardized 

factor 

loading 

Significance 

(P) 

Cronbach’s 

alpha 

AVE 

Communication 

REa 

0.814 

<0.001 

(RE) 

REb 

0.771 

<0.001 

0.76 

52% 

REd 

0.560 

<0.001 

Amount  of 

ATla 

0.875 

<0.001 

surveillance 

ATlc 

0.938 

<0.001 

0.94 

84% 

(ATI) 

ATld 

0.934 

<0.001 

Procurement  lead- 

RBVlc 

0.612 

<0.001 

time  sufficiency 

RBVld 

0.862 

<0.001 

0.81 

61% 

(RBV1) 

RBVle 

0.839 

<0.001 

Internal  customer 

ICCa 

0.909 

<0.001 

commitment 

ICCb 

0.950 

<0.001 

0.90 

76% 

(ICC) 

ICCd 

0.752 

<0.001 

The  final  test  for  discriminant  validity  was  conducted  using  a  comparison 
of  AVE  against  squared  construct  correlation  coefficients.  This  is  a  rigorous  test  for 
discriminant  validity  (Hair  et  ah,  2010),  in  which  two  constructs  can  be  considered 
distinct  from  one  another  if  the  average  variance  extracted  values  for  both  constructs  are 
higher  than  their  squared  correlation  coefficient.  An  examination  of  the  values  in  Table 
29 — in  which  correlations  are  above  the  diagonal,  AVE  values  are  on  the  diagonal  and  in 
bold,  and  squared  correlations  are  below  the  diagonal — revealed  that  the  exogenous 
constructs  are  indeed  distinct;  discriminant  validity  can  therefore  be  assumed. 
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Table  29.  Squared  Exogenous  Construct  Correlation  Matrix  and  AVE 


Variables 

(1) 

(2) 

(3) 

(4) 

(1)  Communication 

0.52 

0.25 

0.41 

0.58 

(2)  Amount  of  surveillance 

0.06 

0.84 

0.10 

0.20 

(3)  Sufficiency  of 

procurement  lead-time 

0.17 

0.01 

0.61 

0.40 

(4)  Internal  customer 
commitment 

0.34 

0.04 

0.16 

0.76 

3.  Endogenous  Measurement  Model 

A  second  assessment  of  model  fit  and  construct  validity  was  conducted  for 
the  endogenous  structures  using  CFA  with  MLE  in  Mplus  Version  6.0.  As  before  in  the 
exogenous  measurement  model,  the  variances  of  latent  constructs  were  held  at  1 ,  while 
regression  weights  between  latent  variables  and  their  respective  factors  were  free 
parameters,  and  all  three  latent  variables  were  permitted  to  covary  freely.  In  the  path 
diagram  of  the  endogenous  measurement  model  (Figure  8),  endogenous  constructs  are 
represented  by  the  Greek  letter  eta  (r|),  and  measurement  error  of  factors  associated  with 
endogenous  constructs  are  labeled  with  the  Greek  letter  epsilon  (s).  The  endogenous 
measurement  model  contains  14  observed  variables,  105  sample  moments,  and  31  free 
parameters.  It  is  therefore  overidentified  with  74  degrees  of  freedom. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


74 


As  before,  fit  of  the  measurement  model  (Table  30)  was  first  examined 
using  the  chi-square  test.  The  chi-square  test  statistic  of  127.63  is  significant  with  a  p 
value  of  less  than  0.001,  indicating  that  fit  may  be  less  than  adequate.  Since  the  chi- 
square  test  is  known  to  penalize  complex  models  and  large  samples,  a  global  assessment 
of  model  fit  was  performed  using  multiple  fit  indices.  The  relative  chi-square  value  of 
1.72  is  less  than  the  threshold  of  3.0  recommended  by  Kline  (1998),  suggesting  adequate 
fit.  Additionally,  CFI  and  NNFI  are  greater  than  the  0.95  minimums  that  were  proposed 
by  Hair  et  al.  (2010),  indicating  good  fit.  Similarly,  the  RMSEA  value  of  0.06  and  the 
SRMR  value  of  0.03  indicate  good  fit  and  are  less  than  the  thresholds  of  0.08  proposed 
by  Hair  et  al.  (2010)  for  each. 


Table  30,  Endogenous  Measurement  Model  Diagnostics 


X!/df 

CFI 

TLI 

RMSEA 

SRMR 

Measurement  model 

1.72 

0.98 

0.98 

0.06 

0.03 

Fitness  criterion 

<3.00 

>0.95 

>0.95 

<0.08 

<0.08 
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An  examination  of  factor  loadings  from  confirmatory  factor  analysis 
(Table  31)  revealed  all  loadings  to  be  above  0.50,  the  minimum  loading  recommended  by 
Hair  et  al.  (2010),  with  the  exception  of  one  item  from  regulatory  and  statutory 
compliance  construct,  which  presented  a  loading  of  0.392.  However,  further  analysis 
reveals  that  the  loading  is  statistically  significant,  all  constructs  exceed  the  minimum 
construct  reliability  alpha  of  0.70,  and  all  constructs,  including  regulatory  and  statutory 
compliance,  exceed  the  minimum  50%  AVE  also  recommended  by  Hair  et  al.  (2010). 
Furthermore,  no  instances  of  negative  error  variance  are  present  (Appendix  I). 


Table  31.  Endogenous  Construct  Reliability  and  Convergent  Validity 


Construct 

Items 

Standardized 

factor 

loadings 

Significance 

ip) 

Cronbach’s 

alpha 

AVE 

Service  quality 

SQa 

0.812 

<0.001 

(SQ) 

SQc 

0.933 

<0.001 

SQd 

0.912 

<0.001 

SQf 

0.726 

<0.001 

SQi 

0.810 

<0.001 

0.96 

76% 

SQ) 

0.909 

<0.001 

SQk 

0.919 

<0.001 

SQm 

0.908 

<0.001 

Regulatory  and 

RSCa 

0.951 

<0.001 

statutory 

RSCc 

0.825 

<0.001 

0.70 

58% 

compliance  (RSC) 

RSCe 

0.391 

<0.001 

Sufficiency  of  the 

RDla 

0.871 

<0.001 

requirement 

RDlb 

0.917 

<0.001 

0.87 

71% 

definition  (RD1) 

RDle 

0.724 

<0.001 
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As  before,  the  test  for  discriminant  validity  of  endogenous  constructs  was 
perfonned  using  a  comparison  of  AVE  against  squared  construct  correlation  coefficients. 
An  examination  of  values  in  Table  32 — in  which  correlations  are  above  the  diagonal, 
AVE  values  are  on  the  diagonal  and  in  bold,  and  squared  correlations  are  below  the 
diagonal — reveals  that  the  exogenous  constructs  are  indeed  distinct,  and  discriminant 
validity  can  also  be  assumed  for  the  endogenous  structures. 


Table  32.  Squared  Endogenous  Construct  Correlation  Matrix  and  AVE 


Variables 

(1) 

(2) 

(3) 

(1)  Service  quality 

0.76 

0.34 

0.46 

(2)  Regulatory  and  statutory 
compliance 

0.12 

0.58 

0.43 

(3)  Sufficiency  of  requirement 
definition 

0.21 

0.18 

0.71 

G.  HYPOTHESIZED  STRUCTURAL  MODEL 

Finally,  the  structural  model  (Figure  9) — containing  all  variables,  factors,  and 
paths  for  the  testing  of  hypotheses — was  analyzed  using  MLE  in  Mplus  Version  6.0.  In 
the  path  diagram,  parameter  estimates  from  exogenous  variables  to  endogenous  variables 
are  represented  by  the  Greek  letter  gamma  (y),  and  parameter  estimates  between 
endogenous  variables  are  represented  by  the  Greek  letter  beta  ((]).  As  is  typical  when 
modeling  with  structural  equations,  exogenous  constructs  were  permitted  to  covary 
freely.  The  structural  model  contains  32  observed  variables,  528  sample  moments,  and 
125  free  parameters  and  is  therefore  overidentified  with  403  degrees  of  freedom.  The 
following  are  structural  equations  for  this  model: 

TJ 1  =  +  /5l^2  +  /S<^3  +  /6X\3  +  /lXl4  +  /9X15  +  (3\T]2  +  jdir/i  +  t^x  (2) 

772  =  /4<^i  +  yio^3  +  y\9^4  +  y\\X\5  +  y\iX\6  +  yisXn  +  y\iX\x  +  f3  3773  +  ^2  (3) 

773  =  +  y\%^4  +  y2X\4  +  ynXi6  +  yuXn  +  yi6Xis  +  £3  (4) 
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Figure  9.  Path  Diagram  of  the  Hypothesized  Structural  Model 
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Fit  of  the  structural  model  (Table  33)  was  assessed  using  the  same  criteria  as 
those  applied  to  the  measurement  models.  The  chi-square  value  of  532.26  is  significant 
with  a  p  value  of  less  than  0.001,  suggesting  that  fit  may  be  less  than  adequate.  However, 
the  relative  chi-square  value  of  1.32  is  less  than  the  3.0  threshold  recommended  by  Kline 
(1998),  suggesting  good  fit.  Since  the  chi-square  test  is  not  a  representative  measure  for 
higher-order  models  (Fornell,  1983),  other  measures  were  again  used.  The  values  for  CFI 
and  TLI  are  both  higher  than  the  0.95  minimums  that  were  proposed  by  Hair  et  al. 
(2010),  suggesting  good  fit.  Similarly,  the  RMSEA  and  SRMR  values  of  0.04  and  0.05 
are  less  than  the  thresholds  proposed  by  Hair  et  al.  (2010),  who  recommended  maximum 
values  of  0.08  for  each.  Overall,  the  structural  model  appears  to  exhibit  good  fit  to  the 
sample  data. 


Table  33.  Structural  Model  Diagnostics 


X!/df 

CFI 

TLI 

RMSEA 

SRMR 

Structural  model 

1.32 

0.97 

0.97 

0.04 

0.05 

Fitness  criterion 

<3.00 

>0.95 

>0.95 

<0.08 

<0.08 

Table  34  displays  the  standardized  loading  estimates  of  construct  items  in  the 
structural  model.  All  items  loaded  above  the  0.50  minimum  recommended  by  Hair  et  al. 
(2010),  with  the  exception  of  one  item  in  the  regulatory  and  statutory  compliance 
construct.  As  previously  discussed,  the  decision  to  retain  this  item  was  made  based  on  a 
review  of  construct  reliability  and  AVE,  which  exceeded  the  recommended  minimums. 

Standardized  path  estimates  for  the  22  hypothesized  relationships  are  displayed  in 
Table  35.  In  reviewing  the  table,  several  of  the  paths  did  not  result  in  statistically 
significant  estimates  because  of  high  p  values  and  low  critical  ratios.  Path  yi,  the 
hypothesized  causal  relationship  from  the  constructs  extent  contractor  defined 
requirements  to  service  quality,  has  a  high  p  value,  indicating  that  the  parameter  estimate 
did  not  differ  significantly  from  0.  Similarly,  the  hypothesized  causal  relationships 
between  the  two  agency  theory  variables  and  service  quality,  represented  by  paths  y5  and 
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76,  were  not  found  to  be  statistically  significant.  Additional  paths  between  sufficient 
procurement  lead-time  (79)  and  internal  customer  commitment  to  regulatory  and  statutory 
compliance  (719)  were  also  not  found  to  be  statistically  significant,  as  were  any  of  the 
paths  stemming  from  the  manifest  competence-based  view  variables.  A  complete 
assessment  of  paths,  including  those  effects  occurring  directly  as  a  result  of  one  causal 
path  and  those  effects  occurring  indirectly  through  one  or  more  mediating  variables,  can 
be  found  in  Table  36.  This  table  also  displays  the  coefficient  of  determination  (R  )  for 
each  endogenous  latent  variable.  For  an  endogenous  (dependent  or  explained)  variable, 
R  is  a  measure  of  the  portion  of  variance  about  the  mean  that  is  explained  by  the 
exogenous  (independent  or  explanatory)  variables  (Hair  et  ah,  2010).  R  for  the 
endogenous  variable  service  quality  was  relatively  high  at  0.655,  indicating  a  high  degree 
of  explanatory  power.  R"  was  greater  than  0.30  for  the  other  two  endogenous  variables, 
regulatory  and  statutory  compliance  and  requirement  definition  sufficiency,  indicating  a 
moderate  degree  of  explanatory  power. 
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Table  34 


Structural  Model  Loading  Estimates 


Parameter 


To 


From 


SQa 

SQc 

SQd 

SQf 

SQi 

SQj 

SQk 

SQm 

RSCa 

RSCc 

RSCe 

REa 

REb 

REd 

ATla 

ATlc 

ATld 

RDla 

RDlb 

RDle 

RBVlc 

RBVld 

RBVle 

ICCa 

ICCb 

ICCd 


SQ 

SQ 

SQ 

SQ 

SQ 

SQ 

SQ 

SQ 

RSC 

RSC 

RSC 

RE 

RE 

RE 

ATI 

ATI 

ATI 

RD1 

RD1 

RD1 

RBV1 

RBV1 

RBV1 

ICC 

ICC 

ICC 


To 

Yi 

Y2 

y3 

y4 

y5 

y6 

y7 

y8 

y9 

Yi 

Yi 

Xi 

X2 

X3 

X4 

X5 

X6 

Yi 

Yj 

Yi 

X7 

X8 

x9 

Xi 

Xi 

Xi 


Model  notation 


From  Path 


ill 

Hi 

ill 

Hi 

ill 

Hi 

ill 

Hi 

ni 

ni 

ni 

& 

^2 

^2 

^2 

H-3 

H-3 

13 

^3 

^3 

^4 

^4 

^4 


kl 

k2 

k3 

k4 

^5 

k7 

k8 

^10 

k|l 

ki2 

^13 

^14 

^15 

^16 

>-tl 

^18 

k39 

^20 

k2i 

’>-11 

k23 

’>-24 

k25 

^26 


Standardized 

estimate 

S.E. 

Critical 

ratio 

P  value 

0.809 

0.023 

34.756 

<0.001 

0.929 

0.010 

90.246 

<0.001 

0.911 

0.012 

74.245 

<0.001 

0.724 

0.032 

22.933 

<0.001 

0.808 

0.023 

34.526 

<0.001 

0.906 

0.013 

71.239 

<0.001 

0.915 

0.012 

77.684 

<0.001 

0.906 

0.013 

70.654 

<0.001 

0.966 

0.030 

31.801 

<0.001 

0.808 

0.033 

24.168 

<0.001 

0.381 

0.059 

6.491 

<0.001 

0.797 

0.031 

25.975 

<0.001 

0.803 

0.030 

26.436 

<0.001 

0.542 

0.051 

10.689 

<0.001 

0.873 

0.018 

48.857 

<0.001 

0.939 

0.013 

74.629 

<0.001 

0.933 

0.013 

72.045 

<0.001 

0.871 

0.023 

38.322 

<0.001 

0.916 

0.020 

45.241 

<0.001 

0.727 

0.059 

21.094 

<0.001 

0.622 

0.046 

13.601 

<0.001 

0.843 

0.031 

27.643 

<0.001 

0.849 

0.030 

28.030 

<0.001 

0.914 

0.016 

57.963 

<0.001 

0.942 

0.014 

66.622 

<0.001 

0.756 

0.030 

24.973 

<0.001 
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Table  35 


Structural  Model  Path  Estimates 


Parameter 
To  From 

SQ  «-  RSC 
SQ  <-  RD1 
RSC  <-  RD1 
SQ  «-  RD2 
RD1  <-  RD2 
SQ  <—  RE 
RSC  «-  RE 
SQ  <-  ATI 
SQ  «-  AT2 
RD1  <-  RBV1 
SQ  «-  RBV1 
RSC  <-  RBV1 
SQ  <-  RBV2 
RSC  <-  RBV2 
RD1  <-  CBV1 
RSC  ^  CBV1 
RD1  CBV2 

RSC  ^  CBV2 
RD1  CBV3 

RSC  ^  CBV3 
RD1  ICC 

RSC  ^  ICC 


Model  notation 

_  Standardized 

To  From  Path  estimate 


S.E. 


Critical 

ratio 


P  value 


111 
Hi 

112 
Hi 
113 
Hi 
112 
Hi 
111 
113 
Hi 
ni 
111 
02 
113 
02 
113 

02 

113 

02 

113 

02 


02 

Pi 

-0.252 

0.074 

-3.406 

0.001 

03 

P2 

0.181 

0.066 

2.716 

0.007 

03 

P3 

0.149 

0.077 

1.937 

0.053 

X[4 

Yi 

0.027 

0.051 

0.527 

0.598 

X]4 

72 

-0.110 

0.058 

-1.884 

0.060 

73 

0.930 

0.079 

11.773 

<0.001 

74 

0.578 

0.083 

6.963 

<0.001 

^2 

7s 

0.048 

0.055 

0.887 

0.375 

A 1 3 

76 

-0.044 

0.053 

-0.827 

0.408 

^3 

77 

0.317 

0.071 

4.454 

<0.001 

^3 

78 

-0.204 

0.068 

-2.993 

0.003 

^3 

79 

-0.027 

0.079 

-0.341 

0.733 

X1S 

Yio 

0.077 

0.052 

1.498 

0.134 

X15 

Yu 

-0.094 

0.059 

-1.599 

0.110 

X 1 6 

712 

0.011 

0.075 

0.149 

0.882 

X|6 

Yl3 

0.038 

0.071 

0.537 

0.591 

X17 

714 

0.044 

0.074 

0.589 

0.566 

X17 

7 15 

-0.067 

0.073 

-0.925 

0.355 

X18 

716 

0.032 

0.060 

0.531 

0.596 

Xlg 

7 17 

0.051 

0.057 

0.901 

0.367 

^4 

YlS 

0.377 

0.065 

5.780 

<0.001 

^4 

719 

-0.039 

0.079 

-0.494 

0.621 
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Table  36. 


Standardized  Direct,  Indirect,  and  Total  Effects 


Dependent  variable  Determinant 

Service  quality  (R2  =  0.655) 

Regulatory  and  statutory  compliance 
Requirement  definition  sufficiency 
Communication 
Amount  of  surveillance 
Extent  PBS  A 

Extent  contractor-defined  requirements 
Procurement  lead-time  sufficiency 
Turnover 

Regulatory  and  statutory  compliance  (R  =  0.388) 

Communication 

Requirement  definition  sufficiency 

Procurement  lead-time  sufficiency 

Turnover 

Buyer  experience 

Buyer  APDP  certification  level 

Buyer  education  level 

Internal  customer  commitment 

Requirement  definition  sufficiency  (R  =0.337) 

Extent  contractor  defined  requirements 
Procurement  lead-time  sufficiency 
Buyer  experience 
Buyer  APDP  certification  level 
Buyer  education  level 
Internal  customer  commitment 


Direct  effect 

Sig. 

Indirect  effects 

Sig. 

Total  effect 

Sig. 

-0.252 

0.001 

-0.252 

0.001 

0.181 

0.007 

-0.038 

0.022 

0.143 

0.029 

0.930 

<  0.001 

-0.145 

0.007 

0.784 

<0.001 

0.048 

0.375 

0.048 

0.375 

-0.044 

0.408 

-0.044 

0.408 

0.027 

0.598 

-0.016 

0.153 

0.011 

0.826 

-0.204 

0.003 

0.052 

0.093 

-0.152 

0.017 

0.077 

0.134 

0.024 

0.147 

0.101 

0.043 

0.578 

<  0.001 

0.578 

<0.001 

0.149 

0.053 

0.149 

0.053 

-0.027 

0.733 

0.047 

0.077 

0.020 

0.789 

-0.094 

0.110 

-0.094 

0.110 

0.038 

0.591 

0.002 

0.882 

0.040 

0.579 

-0.067 

0.355 

0.007 

0.573 

-0.061 

0.408 

0.051 

0.367 

0.005 

0.608 

0.056 

0.329 

-0.039 

0.621 

0.056 

0.067 

0.017 

0.828 

-0.110 

0.060 

-0.110 

0.060 

0.317 

<  0.001 

0.317 

<0.001 

0.011 

0.882 

0.011 

0.882 

0.044 

0.556 

0.044 

0.556 

0.032 

0.596 

0.032 

0.596 

0.377 

<  0.001 

0.377 

<0.001 

Table  37. 


Correlation  Matrix 


Variables 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 

(12) 

(13) 

(1) 

Service  quality 

1.00 

(2) 

Regulatory  and  statutory  compliance 

** 

0.32 

1.00 

(3) 

Communication 

** 

0.78 

** 

0.62 

1.00 

(4) 

Surveillance 

** 

0.23 

** 

0.21 

** 

0.25 

1.00 

(5) 

PBSA 

* 

0.13 

** 

0.17 

** 

0.23 

** 

0.25 

1.00 

(6) 

Contractor-defined  requirements 

0.07 

0.02 

0.06 

0.06 

* 

-0.16 

1.00 

(7) 

Requirement  definition 

** 

0.45 

** 

0.41 

** 

0.59 

* 

0.14 

* 

0.17 

-0.08 

1.00 

(8) 

Procurement  lead-time 

** 

0.19 

** 

0.26 

** 

0.41 

0.10 

* 

0.15 

-0.01 

** 

0.44 

1.00 

(9) 

Turnover 

0.05 

-0.13 

-0.05 

-0.05 

-0.06 

-0.07 

-0.02 

0.01 

1.00 

(10) 

Buyer  experience 

0.10 

0.02 

0.01 

0.00 

0.02 

-0.08 

-0.02 

** 

-0.19 

0.01 

1.00 

(ID 

APDP  level 

0.05 

-0.04 

0.04 

0.07 

0.03 

-0.03 

-0.04 

** 

-0.21 

0.12 

** 

0.60 

1.00 

(12) 

Education  level 

-0.05 

0.02 

-0.01 

0.06 

-0.01 

-0.04 

0.02 

-0.11 

0.04 

-0.05 

* 

0.14 

1.00 

(13) 

Internal  customer  commitment 

** 

0.42 

** 

0.36 

** 

0.58 

** 

0.26 

** 

0.20 

0.09 

** 

0.48 

** 

0.40 

-0.05 

0.00 

-0.10 

0.02 

1.00 

Note.  *p  <0.05,  **p  <0.01. 


H. 


TRIMMED  STRUCTURAL  MODEL 


In  the  interest  of  achieving  a  more  parsimonious  model,  paths  with  non¬ 
significant  relationships  were  considered  for  potential  trimming.  This  occurred  in  an 
iterative  process  in  which  paths  presenting  critical  ratios  of  less  than  1  were  considered  as 
candidates  for  removal.  Similarly,  variables  and  constructs  were  trimmed  if  they 
presented  no  statistically  significant  paths  and,  therefore,  had  little  relevance  to  the 


7)  i  =  73^1 +  75^2 +  78^3 +  76X13 +  71X14 +  79X15  +  j3ir/2  +  j3ir]3  +  £\ 

7)2  =  74^1  +  7lo4"3  +  719^4+71  lX~15  +  71 3X1 6  +  71 5X1 7  +  71 7X1 8  +  P  37)3  +  7^2 
773  =  77l^3  +  7181^4  +  72X14  +  712X16  +  7l4Xl7  +  7I6X18  +  £3 


Figure  10.  Path  Diagram  of  the  Trimmed  Model 


Fit  of  this  respecified  model  (Table  38)  was  assessed  using  the  same  criteria  as 
those  applied  to  the  previous  measurement  models  and  the  hypothesized  structural  model. 
The  chi-square  value  of  360.61  is  significant  with  a  p  value  of  less  than  0.001.  While  this 
suggests  that  fit  may  be  less  than  adequate,  the  relative  chi-square  value  is  1.41  and  less 
than  the  3.00  recommended  threshold,  indicating  good  fit  to  the  sample  data.  As  before, 
a  global  assessment  of  fit  was  also  performed  using  CFI,  TLI,  RMSEA,  and  SRMR  since 
the  chi-square  test  is  not  representative  of  higher  order  models  and  tends  to  penalize  for 
complexity  and  larger  samples.  The  values  for  CFI  and  TLI  are  both  higher  than  the  0.95 
minimums  that  were  proposed  by  Hair  et  al.  (2010),  suggesting  good  fit.  Similarly,  the 
RMSEA  and  SRMR  values  of  0.04  and  0.06  are  less  than  the  thresholds  proposed  by  Hair 
et  al.  (2010),  who  recommended  maximum  values  of  0.08  each.  Overall,  the  trimmed 
structural  model  appears  to  fit  well  to  the  sample  data. 


Table  38.  Trimmed  Structural  Model  Diagnostics 


X!/df 

CFI 

TLI 

RMSEA 

SRMR 

Structural  model 

1.41 

0.98 

0.98 

0.04 

0.06 

Fitness  criterion 

<3.00 

>0.95 

>0.95 

<0.08 

<0.08 

A  review  of  the  standardized  path  and  loading  estimates  in  Table  39  reveals  all 
loadings  to  be  significant  with  p  values  less  than  0.001.  Additionally,  path  estimates  for 
all  hypothesized  relationships  are  either  significant  with  a  p  value  of  less  than  0.05, 
marginally  significant  with  a  p  value  greater  than  0.05  but  less  than  0.10,  or  meet  the 
inclusion  criteria  of  having  a  critical  ratio  greater  than  1.0.  Standardized  direct,  indirect, 
and  total  effects  are  again  displayed  (Table  40),  where  a  total  effect  is  the  summation  of 
the  direct  effect  and  any  indirect  effects.  The  coefficient  of  determination  (R  )  for  the 
endogenous  variable  service  quality  was  relatively  high,  at  0.651,  indicating  a  high 
degree  of  explanatory  power.  R"  was  greater  than  0.30  for  the  other  two  endogenous 
variables,  regulatory  and  statutory  compliance  and  requirement  definition  sufficiency, 
indicating  a  moderate  degree  of  explanatory  power. 
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Table  39.  Trimmed  Structural  Model  Loading  and  Path  Estimates 


Parameter 

Model  notation 

Standardized 

estimate 

S.E. 

Critical 

ratio 

P  value 

To 

From 

To 

From 

Path 

SQa 

«- 

SQ 

Y, 

ill 

0.809 

0.023 

34.783 

<0.001 

SQc 

<- 

SQ 

y2 

<- 

ill 

^2 

0.929 

0.010 

90.137 

<0.001 

SQd 

<- 

SQ 

y3 

<- 

ill 

^3 

0.911 

0.012 

74.099 

<0.001 

SQf 

<- 

SQ 

y4 

<- 

Hi 

X.4 

0.723 

0.032 

22.923 

<0.001 

SQi 

<- 

SQ 

y5 

<- 

ill 

^5 

0.808 

0.023 

34.476 

<0.001 

SQj 

<- 

SQ 

y6 

<- 

ill 

^6 

0.906 

0.013 

70.980 

<0.001 

SQk 

<- 

SQ 

y7 

<- 

ill 

^7 

0.915 

0.012 

77.652 

<0.001 

SQm 

<- 

SQ 

y8 

<- 

ill 

0.906 

0.013 

70.703 

<0.001 

RSCa 

<- 

RSC 

y9 

<- 

Il2 

^<9 

0.963 

0.030 

32.295 

<0.001 

RSCc 

«- 

RSC 

Yio 

«- 

112 

kio 

0.811 

0.033 

24.634 

<0.001 

RSCe 

<- 

RSC 

Y„ 

<- 

kll 

0.383 

0.058 

6.539 

<0.001 

REa 

<- 

RE 

x, 

«- 

^12 

0.797 

0.031 

25.962 

<0.001 

REb 

<- 

RE 

x2 

<- 

^13 

0.807 

0.030 

26.860 

<0.001 

REd 

«- 

RE 

x3 

«- 

kl4 

0.537 

0.051 

10.527 

<0.001 

RDla 

<- 

RD1 

Y12 

<- 

'T 

^18 

0.870 

0.023 

38.276 

<0.001 

RDlb 

<- 

RD1 

y13 

«- 

113 

^19 

0.916 

0.020 

45.370 

<0.001 

RDle 

<- 

RD1 

Y,4 

<- 

H3 

k2o 

0.727 

0.034 

21.101 

<0.001 

RBVlc 

«- 

RBV1 

X7 

«- 

^3 

k2i 

0.620 

0.046 

13.454 

<0.001 

RBVld 

<- 

RBV1 

x8 

<- 

^3 

k22 

0.846 

0.031 

27.358 

<0.001 

RBVle 

<- 

RBV1 

X9 

<- 

^3 

k23 

0.848 

0.031 

27.523 

<0.001 

ICCa 

«- 

ICC 

X,o 

<- 

k24 

0.911 

0.016 

56.898 

<0.001 

ICCb 

<- 

ICC 

X„ 

<- 

k25 

0.945 

0.014 

67.168 

<0.001 

ICCd 

<- 

ICC 

Xi2 

«- 

^4 

k26 

0.754 

0.030 

24.814 

<0.001 

SQ 

<- 

RSC 

Ill 

<- 

d2 

Pi 

-0.240 

0.072 

-3.318 

0.001 

SQ 

<- 

RD1 

111 

<- 

113 

P2 

0.172 

0.066 

2.605 

0.009 

RSC 

<- 

RD1 

112 

<- 

'1 3 

P3 

0.140 

0.072 

1.953 

0.051 

RD1 

<- 

RD2 

Ill 

<- 

X,4 

72 

-0.112 

0.058 

-1.934 

0.053 

SQ 

<- 

RE 

Ill 

<- 

73 

0.930 

0.072 

12.919 

<0.001 

RSC 

<- 

RE 

H2 

<- 

74 

0.539 

0.067 

8.062 

<0.001 

RD1 

<- 

RBV1 

H3 

<- 

^3 

77 

0.299 

0.068 

4.378 

<0.001 

SQ 

<- 

RBV1 

111 

<- 

^3 

7s 

-0.199 

0.066 

-3.007 

0.003 

SQ 

<- 

RBV2 

Ill 

<- 

X15 

7io 

0.078 

0.051 

1.511 

0.131 

RSC 

<- 

RBV2 

H2 

<- 

X15 

7n 

-0.101 

0.058 

-1.743 

0.081 

RD1 

«- 

ICC 

113 

«- 

^4 

718 

0.380 

0.064 

5.921 

<0.001 
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Table  40. 


Standardized  Direct,  Indirect,  and  Total  Effects  for  Trimmed  Model 


Dependent  variable 

Determinant 

Direct  effect 

Sig. 

Indirect  effects 

Sig. 

Total  effect 

Sig. 

Service  quality  (R2  =  0.651) 

Regulatory  and  statutory  compliance 

-0.240 

0.001 

-0.240 

0.001 

Requirement  definition  sufficiency 

0.172 

0.009 

-0.034 

0.071 

0.138 

0.033 

Communication 

0.930 

<  0.001 

-0.130 

0.006 

0.800 

<  0.001 

Procurement  lead-time  sufficiency 

-0.199 

0.003 

0.041 

0.062 

-0.158 

0.012 

Turnover 

0.078 

0.131 

0.024 

0.123 

0.102 

0.041 

Regulatory  and  statutory  compliance  (R2  =  0.378) 

Communication 

0.539 

<  0.001 

0.539 

<  0.001 

Requirement  definition  sufficiency 

0.140 

0.051 

0.140 

0.051 

Turnover 

-0.101 

0.081 

-0.101 

0.081 

Requirement  definition  sufficiency  (R2  =  0.330) 

Extent  contractor-defined  requirements 

-0.101 

0.081 

-0.101 

0.081 

Procurement  lead-time  sufficiency 

0.299 

<  0.001 

0.299 

<  0.001 

Internal  customer  commitment 

0.380 

<  0.001 

0.380 

<  0.001 

I. 


COMPARISON  OF  COMPETING  MODELS 


Although  the  trimmed  structural  model  is  more  parsimonious  than  the 
hypothesized  structural  model  and  exhibits  a  similar  goodness  of  fit,  further  empirical 
analysis  is  required  to  properly  compare  models  and  to  determine  the  most  preferred 
model.  Competing  models  are  typically  categorized  as  either  nested  models  or  non¬ 
nested  models.  Models  are  considered  to  be  to  be  nested  when  all  of  one  model’s  free 
parameters  are  a  subset  of  another  model’s  free  parameters  (Hoyle  &  Panter,  1995).  In 
such  situations,  Hoyle  and  Panter  (1995)  argued  for  comparison  using  absolute  fit 
indices,  which  are  based  on  chi-square  differences.  Alternatively,  non-nested  models 
have  different  numbers  of  total  parameters,  meaning  that  the  number  of  free  parameters, 
or  those  parameters  being  estimated,  plus  the  number  of  constrained  parameters  differs 
between  the  models.  In  the  comparison  of  non-nested  models,  the  most  appropriate 
statistics  are  the  Akaike  Information  Criterion  (AIC)  and  the  Bayesian  Information 
Criterion  (BIC).  Both  AIC  and  BIC  provide  an  adjustment  for  the  number  of  free 
parameters  being  estimated,  while  BIC  penalizes  more  than  AIC  for  model  complexity 
(Byrne,  2010;  Hu  &  Bender,  1995).  In  both  cases,  the  preferred  model  is  the  one  with 
the  lowest  information  criterion  value. 

The  models  presented  in  this  study  are  considered  to  be  non-nested  because 
several  latent  and  manifest  exogenous  variables  have  been  removed  from  the 
hypothesized  model.  As  such,  it  is  appropriate  to  identify  the  best  model  through  a 
comparison  of  AIC  and  BIC,  which  were  obtained  from  the  SEM  output  produced  by 
Mplus  Version  6.0.  As  seen  in  Table  41,  the  trimmed  model  presents  lower  values  for 
both  criteria  and  can  therefore  be  considered  the  most  preferred  model  of  the  two. 


Table  41,  Information  Criteria  for  Competing  Models 


Structural  Model 

2 

5C 

P-value 

df 

AIC 

BIC 

Hypothesized 

532.26 

<0.001 

403 

22702.24 

23154.73 

Trimmed 

360.61 

<0.001 

256 

16781.71 

17091.48 
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J. 


FURTHER  ANALYSIS 


Additional  analysis  of  the  relationships  between  variables  can  often  provide 
further  insight  when  previous  analysis  can  be  extended  or  when  results  can  be  more  fully 
explored.  This  section  will  present  results  from  further  analytic  research  and  will  employ 
a  variety  of  statistical  methods,  such  as  SEM,  linear  and  non-linear  regression  analysis, 
analysis  of  variance  (ANOVA),  and  multivariate  analysis  of  variance  (MANOVA). 
Additional  tests  involving  personnel  turnover,  buyer  experience,  APDP  level,  and 
education  level,  as  well  as  tests  of  the  factors  affecting  the  assignment  of  personnel  to 
services  acquisitions,  are  perfonned. 

1.  Turnover  of  Acquisition  Personnel 

Neither  the  hypothesized  relationship  between  the  turnover  of  acquisition 
personnel  and  service  quality  nor  the  hypothesized  relationship  between  the  turnover  of 
acquisition  personnel  and  compliance  with  regulations  and  statutes  were  found  to  be 
highly  significant  in  the  trimmed  model;  p  values  for  both  path  coefficients  were  slightly 
greater  than  0.05.  To  further  analyze  these  relationships,  three  modifications  were  applied 
to  the  turnover  variable,  RBV2,  which  represented  percent  turnover  as  a  ratio  of  the 
number  of  times  acquisition  personnel  had  turned  over  to  the  number  of  acquisition 
personnel  assigned.  First,  an  additional  variable  was  created  as  a  high-low  binary 
variable,  PTbin,  where  a  cut  in  the  data  occurred  about  the  median  (1.00).  Since  the 
values  of  RBV2  were  calculated  as  ratios,  responses  were  coded  in  PTbin  with  a  value  of 
1  if  the  number  of  times  that  acquisition  personnel  turned  over  was  greater  than  the 
median  of  1.00  (n  =  130).  All  other  responses  were  coded  with  a  value  of  0  (n  =  1 10). 
As  such, 

Proposition  1:  The  level  of  service  quality  and  the  amount  of 
compliance  with  regulations  and  statues  differ  between 
acquisitions  where  turnover  ratios  are  high  and  acquisitions  where 
turnover  ratios  are  low. 
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Next,  the  ratio  of  personnel  turnover  was  modified  to  represent  the 
annualized  percent  turnover  ( APT)  by  accounting  for  the  duration  of  the  contract.  As 
discussed  in  Chapter  III,  respondents  were  asked  for  the  amount  of  time  that  had  passed 
since  contract  award.  While  the  previous  SEM  analysis  assumed  that  effects  due  to 
personnel  turnover  were  unrelated  to  time,  this  new  variable  considers  that  the 
relationships  between  turnover  and  contract  outcomes  were  dependent  on  the  duration  in 
which  the  turnover  occurred.  Therefore, 

Proposition  2:  The  higher  the  annualized  percent  turnover,  the 
lesser  the  service  quality  and  compliance  with  regulations  and 
statutes. 

Similar  to  the  previous  binary  transformation,  APT  was  also  modified  as  a 
high-low  binary  variable  where  a  cut  in  the  data  occurred  about  APT’s  median  (0.42). 
Within  this  new  binary  variable,  APTbin,  the  cases  in  which  APT  was  less  than  42%  were 
coded  with  a  value  of  0  (n  =  122),  and  the  cases  in  which  APT  was  greater  than  or  equal 
to  42%  were  coded  with  a  value  of  1  (n  =  118).  As  such, 

Proposition  3:  The  level  of  service  quality  and  the  amount  of 
compliance  with  regulations  and  statues  differ  between 
acquisitions  where  annualized  turnover  ratios  are  high  and 
acquisitions  where  annualized  turnover  ratios  are  low. 

Table  42  contains  descriptive  statistics  for  all  measures  of  turnover  that 
are  used  in  this  analysis.  Additionally,  values  for  service  quality  and  compliance  are 
treated  in  ANOVAs  and  regression  analysis  as  summations  of  the  scale  items  used  in 
CFA  and  SEM  for  each.  Descriptive  statistics  for  these  summated  scales  are  also 
included  in  the  table. 
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Table  42.  Descriptive  Statistics  for  Analysis  of  Personnel  Turnover 


Variable 

N 

Min. 

Max. 

Mean 

Median 

St.  Dev. 

Percent  turnover 
(. RBV2 ) 

240 

0.00 

7.00 

1.22 

1.00 

1.17 

Binary  percent  turnover 
(PTbin) 

240 

0.00 

1.00 

0.54 

1.00 

0.50 

Annualized  percent  turnover 
(APT) 

240 

0.00 

12.00 

0.68 

0.42 

1.09 

Binary  annualized  percent 
turnover  (APTbin ) 

240 

0.00 

1.00 

0.49 

1.00 

0.50 

Summated  scale  of  service 
quality  (SQ) 

240 

21.00 

56.00 

45.21 

48.00 

8.68 

Summated  scale  of 
compliance  (RSC) 

240 

4.00 

21.00 

17.22 

18.00 

3.30 

Proposition  1  was  tested  first  using  a  MANOVA  in  PASW  Statistics  18.0 
to  compare  the  means  of  service  quality  and  regulatory  and  statutory  compliance  between 
the  two  groups,  low  percent  turnover  and  high  percent  turnover.  Test  statistics  for  Pillai’s 
Trace,  Wilks’  Lambda,  Hotelling’s  Trace,  and  Roy’s  Largest  Root  were  statistically 
significant  with  p  values  of  less  than  0.05.  These  results  indicate  that  there  is  a 
significant  effect  of  the  high/low  percent  turnover  variable  on  service  quality  and 
regulatory  and  statutory  compliance  when  the  two  dependent  variables  are  considered  as 
a  group.  These  group  differences  were  further  explored  using  univariate  F  tests.  As  seen 
in  Table  43,  there  is  a  significant  difference  in  the  means  between  the  two  groups  of 
regulatory  and  statutory  compliance  but  no  statistically  significant  difference  in  the 
means  between  the  two  groups  of  service  quality.  As  such,  additional  testing  using  a 
linear  regression  analysis  of  regulator  and  statutory  compliance  on  PTbin  was  appropriate 
to  estimate  the  coefficient  size  and  direction  (Table  44).  The  resulting  estimate  was 
statistically  significant  with  a  standardized  estimate  of -0.175,  indicating  that  compliance 
is  less  on  those  services  acquisitions  in  which  acquisition  personnel  turnover  is  greater 
than  or  equal  to  100%. 
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Table  43.  Univariate  F  Tests  Between  Groups  by  High-Low  Turnover  Ratios 


Dependent  variables 

Sum  of 

squares 

df 

Mean 

square 

F 

Sig. 

RSC 

Between  groups 

79.23 

1 

79.23 

7.48 

0.01 

Within  groups 

2522.07 

238 

10.60 

Total 

2601.30 

239 

SQ 

Between  groups 

27.36 

1 

27.36 

0.36 

0.55 

Within  groups 

17990.80 

238 

75.59 

Total 

18018.16 

239 

Table  44. 

Regression  of  Compliance  on  High  Percent  Turnover 

Unstandardized 

Standardized 

coefficients 

coefficients 

Variable 

B 

S.E. 

P 

t 

Sig. 

(Constant) 

17.845 

0.310 

57.496 

<0.001 

PTbin 

-1.153 

0.422 

-0.175 

-2.734 

0.007 

Note.  Dependent  Variable:  RSC  (R2=0.03) 


SEM  analysis  was  performed  in  Mplus  Version  6.0  to  test  Proposition  2, 
where  RB  V2  was  removed  from  the  trimmed  structural  model  and  replaced  with  APT  in 
an  alternative  model.  As  before,  the  model  remains  overidentified  with  256  degrees  of 
freedom.  The  results  of  this  analysis  can  be  found  in  Figure  1 1 ,  which  contains  the  path 
diagram  and  parameter  coefficients,  and  in  Table  45,  which  contains  fit  diagnostics  and 
all  parameter  estimates.  Although  the  model’s  chi-square  value  of  352.81  was  significant 
with  a  p  value  of  less  than  0.001,  a  global  assessment  of  fit  measures  once  again  indicated 
good  fit.  However,  neither  the  path  estimate  from  APT  to  service  quality  nor  the  path 
estimate  from  APT  to  regulatory  and  statutory  compliance  were  statistically  significant, 
indicating  that  a  linear  relationship  did  not  exist  between  annualized  percent  turnover  and 
contract  outcomes. 
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Figure  11.  Path  Diagram  of  the  Alternative  Model 
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Finally,  Proposition  3  was  tested  using  a  MANOVA  in  PASW  Statistics 
18.0  to  compare  the  means  of  service  quality  and  regulatory  and  statutory  compliance 
between  the  two  groups,  low  annualized  percent  turnover  and  high  annualized  percent 
turnover.  Test  statistics  for  Pillai’s  Trace,  Wilks’  Lambda,  Hotelling’s  Trace,  and  Roy’s 
Largest  Root  were  statistically  significant  with  p  values  of  less  than  0.05.  These  results 
indicate  that  there  is  a  significant  effect  of  the  high-low  annualized  percent  turnover 
variable  on  service  quality  and  regulatory  and  statutory  compliance  when  the  two 
dependent  variables  are  considered  as  a  group.  The  group  differences  were  explored 
further  using  univariate  F  tests.  As  seen  in  Table  46,  there  is  a  statistically  significant 
difference  in  the  means  between  the  two  groups  of  regulatory  and  statutory  compliance 
but  no  statistically  significant  difference  in  the  means  between  the  two  groups  of  service 
quality.  As  such,  additional  testing  using  a  linear  regression  analysis  of  regulatory  and 
statutory  compliance  on  PTbin  was  appropriate  to  estimate  the  coefficient  size  and 
direction  (Table  47).  The  resulting  estimate  was  statistically  significant  with  a 
standardized  estimate  of -0.190,  indicating  that  compliance  was  lesser  on  those  services 
acquisitions  in  which  acquisition  personnel  turnover  was  greater  than  or  equal  to  42% 
annually. 


Table  46.  Univariate  F  Tests  Between  Groups  by  High/Low  APT 


Dependent  variables 

Sum  of 

squares 

df 

Mean 

square 

F 

Sig. 

RSC 

Between  groups 

92.92 

1 

93.92 

8.92 

<0.01 

Within  groups 

2507.37 

238 

10.54 

Total 

2601.30 

239 

SQ 

Between  groups 

17.15 

1 

17.15 

0.23 

0.63 

Within  groups 

18001.01 

238 

75.63 

Total 

18018.16 

239 

ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


96 


Table  47.  Regression  of  Compliance  on  High/Low  APT 


Unstandardized 

Standardized 

coefficients 

coefficients 

Variable 

B 

S.E. 

P 

t 

Sig. 

(Constant) 

17.836 

0.294 

60.696 

<0.001 

APTbin 

-1.251 

0.419 

-0.190 

-2.986 

0.003 

Note.  Dependent  Variable:  RSC  (R"=0.04) 


3.  APDP  Certification  Level 

In  the  hypothesized  structural  model,  the  exogenous  variable  representing 
the  contract  administrator’s  APDP  certification  level  was  trimmed  because  it  did  not 
produce  significant  path  estimates  to  the  variables  representing  requirement  definition 
sufficiency  or  regulatory  and  statutory  compliance.  As  such,  further  analysis  using  a 
MANOVA  was  warranted  to  test  for  differences  in  the  means  between  groups.  As 
mentioned  in  the  literature  review,  contracting  personnel  can  be  awarded  three  different 
levels  of  APDP  certification;  however,  not  all  personnel  possess  such  certification. 
Therefore,  four  APDP  categories  are  available:  APDP  Level  I  (n  =  44),  APDP  Level  II  (n 
=  88),  APDP  Level  III  (n  =  61),  and  no  APDP  Certification  (n  =  47).  A  MANOVA  was 
first  performed  in  PASW  Statistics  18.0  to  test  for  multivariate  differences  in  group 
means.  Test  statistics  for  Pillai’s  Trace,  Wilks’  Lambda,  Hotelling’s  Trace,  and  Roy’s 
Largest  Root  were  statistically  significant  with  p  values  of  less  than  0.05.  These  results 
indicate  that  there  is  a  significant  effect  of  buyer  APDP  level  on  requirement  definition 
sufficiency  and  on  regulatory  and  statutory  compliance,  when  the  two  dependent 
variables  are  considered  as  a  group.  Possible  group  differences  were  explored  further 
using  univariate  F  tests  on  the  two  dependent  variables  between  the  four  APDP  groups. 
The  analysis  resulted  in  a  significant  F  test  statistic  on  requirement  definition  sufficiency 
and  a  marginally  significant  F  test  statistic  on  regulatory  and  statutory  compliance,  as 
seen  in  Table  48. 
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Table  48.  Univariate  F  Tests  of  Outcomes  Between  Groups  of  APDP  Level 


Dependent  variables 

Sum  of 

squares 

df 

Mean 

square 

F 

Sig. 

RSC 

Between  groups 

71.02 

3 

23.67 

2.21 

0.09 

Within  groups 

2530.28 

236 

10.72 

Total 

2601.30 

239 

RD1 

Between  groups 

223.99 

3 

74.66 

4.79 

<0.01 

Within  groups 

3679.81 

236 

15.59 

Total 

3903.80 

239 

Because  the  results  of  the  F  tests  justified  further  investigation,  and 
because  the  inclusion  of  more  than  two  groups  permitted  post-hoc  analysis,  Scheffe’s  test 
was  also  perfonned.  Post-hoc  tests  can  isolate  the  groups  that  differ.  Whereas  there  are 
many  post-hoc  tests  available,  the  Scheffe  test  is  appropriate  when  pairwise  contrasts  are 
not  possible  (i.e.,  group  sizes  differ)  and  there  are  more  than  three  means.  Kirk  (1995) 
stated  that  this  test  -controls  the  Type  I  error  rate  at  less  than  alpha  for  the  infinite 
number  of  contrasts  that  can  be  perfonned”  (p.  154),  but  sacrifices  some  statistical  power 
to  detect  differences.  In  the  test  results  (Table  49),  Column  I  represents  an  APDP  group 
of  the  dependent  variable  of  which  the  mean  is  compared  to  other  APDP  groups  in 
Column  J.  Only  one  statistically  significant  difference  was  found  between  means  of 
APDP  groups;  the  mean  reported  level  of  requirement  definition  sufficiency  by  APDP 
Level  II  contract  administrators  (16.63  on  a  summated  scale  using  three  7-point  items) 
was  more  than  that  reported  by  APDP  Level  III  contract  administrators  (14.20),  and  the 
difference  of  2.43  was  statistically  significant  with  a  p  value  of  less  than  0.01. 
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Table  49.  Scheffe’s  Test  for  Comparison  of  Means  by  APDP  Level 


Dependent 

variables 

CBV2 

Mean 

difference 

(I-J) 

S.E. 

Sig. 

95% 

C.I. 

(I) 

(J) 

Lower 

bound 

Upper 

bound 

RSC 

0 

1 

1.15 

0.69 

0.43 

-0.79 

3.08 

2 

-0.25 

0.59 

0.98 

-1.92 

1.41 

3 

0.67 

0.64 

0.77 

-1.11 

2.46 

1 

0 

-1.15 

0.69 

0.43 

-3.08 

0.79 

2 

-1.40 

0.60 

0.15 

-3.10 

0.30 

3 

-0.47 

0.65 

0.91 

-2.30 

1.35 

2 

0 

0.25 

0.59 

0.98 

-1.41 

1.92 

1 

1.40 

0.60 

0.15 

-0.30 

3.10 

3 

0.93 

0.55 

0.41 

-0.61 

2.46 

3 

0 

-0.67 

0.64 

0.77 

-2.46 

1.11 

1 

0.47 

0.65 

0.91 

-1.35 

2.30 

2 

-0.93 

0.55 

0.41 

-2.46 

0.61 

RD1 

0 

1 

-0.26 

0.83 

0.99 

-2.59 

2.08 

2 

-1.54 

0.71 

0.20 

-3.55 

0.47 

3 

0.89 

0.77 

0.72 

-1.27 

3.05 

1 

0 

0.26 

0.83 

0.99 

-2.08 

2.59 

2 

-1.28 

0.73 

0.38 

-3.34 

0.77 

3 

1.14 

0.78 

0.54 

-1.05 

3.34 

2 

0 

1.54 

0.71 

0.20 

-0.47 

3.55 

1 

1.28 

0.73 

0.38 

-0.77 

3.34 

3 

2.43 

0.66 

<0.01 

0.58 

4.28 

3 

0 

-0.89 

0.77 

0.72 

-3.05 

1.27 

1 

-1.14 

0.78 

0.54 

-3.34 

1.05 

2 

-2.43 

0.66 

<0.01 

-4.28 

-0.58 

4.  Buyer  Education  Level 

A  similar  approach  to  analysis  was  taken  to  explore  differences  in  means 
by  groups  of  varying  buyer  education  level  ( CBV3 ),  which  did  not  produce  significant 
path  estimates  in  SEM  and  was  trimmed  from  the  hypothesized  model.  A  MANOVA 
was  perfonned  on  requirement  definition  sufficiency  and  regulatory  and  statutory 
compliance  by  groups  of  four  education  levels:  high  school  or  general  equivalency 
diploma  (n  =  29),  associate’s  degree  (n  =  104),  bachelor’s  degree  (n  =  28),  and  master’s 
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degree  or  higher  (n  =  79).  None  of  the  test  statistics  for  Pillai’s  Trace,  Wilks’  Lambda, 
Hotelling’s  Trace,  or  Roy’s  Largest  Root  were  statistically  significant,  indicating  that 
group  means  did  not  differ.  Additional  results  from  univariate  F  tests  (Table  50) 
confirmed  this  finding;  no  statistically  significant  differences  in  means  between  groups 
were  found  for  either  dependent  variable. 

Table  50.  Univariate  F  Tests  Between  Groups  of  Education  Level 


Dependent  variables 

Sum  of 

squares 

df 

Mean 

square 

F 

Sig. 

RSC 

Between  groups 

2.37 

3 

0.79 

0.07 

0.98 

Within  groups 

2598.93 

236 

11.01 

Total 

2601.30 

239 

RD1 

Between  groups 

3.43 

3 

1.14 

0.07 

0.98 

Within  groups 

3900.37 

236 

16.53 

Total 

3903.70 

239 

5.  Buyer  Experience 

Further  analysis  was  also  performed  to  explore  potential  relationships 
between  buyer  experience  and  service  quality,  regulatory  and  statutory  compliance,  and 
requirement  definition  sufficiency.  First,  a  MANOVA  was  performed  on  the  latter  three 
variables  when  grouped  using  a  binary  high-low  measure  of  experience,  where  the  cut  in 
data  occurred  about  the  median  of  experience  reported  (8.5  years).  In  the  high-low 
variable  for  buyer  experience,  the  cases  in  which  respondents  had  less  than  8.5  years  of 
experience  were  coded  with  a  value  of  1  (n  =  120),  and  all  other  cases  were  coded  with  a 
value  of  0  (n  =  120).  None  of  the  test  statistics  for  Pillai’s  Trace,  Wilks’  Lambda, 
Hotelling’s  Trace,  or  Roy’s  Largest  Root  were  statistically  significant,  indicating  that 
multivariate  group  means  did  not  differ.  However,  although  univariate  F  tests  were  also 
not  significant  for  any  of  the  dependent  variables  (Table  51),  the  test  result  for  service 
quality  indicated  that  additional  analysis  may  be  warranted  based  on  the  value  of  the  F 
statistic. 
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Table  51.  Univariate  F  Tests  Between  High  and  Low  Groups  of  Experience 


Dependent  variables 

Sum  of 

squares 

df 

Mean 

square 

F 

Sig. 

RSC 

Between  groups 

1.20 

1 

1.20 

0.11 

0.74 

Within  groups 

2600.09 

238 

10.93 

Total 

2601.30 

239 

RD1 

Between  groups 

0.94 

1 

0.94 

0.06 

0.81 

Within  groups 

3902.86 

238 

16.40 

Total 

3903.80 

239 

SQ 

Between  groups 

97.54 

1 

97.54 

1.30 

0.26 

Within  groups 

17920.63 

238 

75.30 

Total 

18018.16 

239 

As  such,  buyer  experience  responses  were  partitioned  into  three  groups: 
low,  medium,  and  high.  Low-experience  buyers  were  considered  to  be  those  respondents 
with  five  or  less  years  of  contract  administration  experience  (n  =  91).  Medium- 
experience  buyers  were  considered  to  be  those  respondents  with  between  5  and  1 5  years 
of  experience  (n  =  76).  Finally,  high-experience  buyers  were  those  respondents  with 
more  than  15  years  of  experience  (n  =  73).  Two  binary  variables  were  created,  one  for 
medium-experience  and  another  for  high-experience.  Service  quality,  compliance,  and 
requirement  definition  sufficiency  were  regressed  individually  on  the  two  variables; 
regression  results  are  located  in  Table  52.  Only  the  regression  of  service  quality 
exhibited  any  level  of  statistical  significance;  both  binary  variables  were  moderately 
significant  with  coefficient  p  values  less  than  0.10.  However,  the  sizes  of  the  estimates 
were  nearly  identical  between  the  two  variables,  indicating  that  an  additional  high-low 
analysis  might  be  appropriate  with  the  medium  and  high  responses  combined.  For  ease 
of  interpretation,  this  additional  regression  of  service  quality  was  only  performed  on  a 
binary  variable  representing  low  experience,  since  medium  and  high  experience  would  be 
effectively  consolidated  in  the  constant  of  the  regression  equation.  This  regression 
analysis  (Table  53)  resulted  in  the  estimation  of  a  statistically  significant  non-positive 
path  coefficient  for  the  binary  variable  representing  low  buyer  experience.  Therefore,  it 
is  concluded  that  low  buyer  experience  (five  years  or  less)  is  associated  with  lower  levels 
of  service  quality. 
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Table  52.  Regressions  on  Buyer  Experience 


Variable 

Unstandardized 

coefficie  nts 

B  S.E. 

Standardized 

coefficients 

P 

t 

Sig.  ip ) 

Regression  of  service  quality  (R2  =  0.02) 

(Constant) 

43.780  0.906 

48.303 

<  0.001 

Experience  Mod. 

2.299  1.344 

0.123 

1.711 

0.088 

Experience  High 

2.316  1.359 

0.123 

1.705 

0.090 

Regression  of  compliance  with  regulations  and  statutes  (R2  <  0.01) 

(Constant) 

17.000  0.347 

49.048 

<  0.001 

Experience  Mod. 

0.500  0.514 

0.071 

0.973 

0.331 

Experience  High 

0.205  0.520 

0.029 

0.396 

0.693 

Regression  of  requirement  definition  sufficiency  (R2  <  0.01) 

(Constant) 

15.516  0.425 

36.481 

<  0.001 

Experience  Mod. 

0.036  0.630 

0.004 

0.057 

0.954 

Experience  High 

-0.188  0.638 

-0.021 

-0.294 

0.769 

Unstandardized 

Standardized 

Coefficients 

Coefficients 

Variable 

B 

S.E. 

P 

t 

Sig.  ip ) 

(Constant) 

46.087 

0.707 

65.202 

<0.001 

Experience  Low 

-2.307 
.  . .  - 11 

1.148 

-0.129 

-2.010 

0.046 

Note:  Dependent  Variable:  SQ  (R~=  0.0 1 ) 


6.  Assignment  of  Contract  Administration  Personnel 

Finally,  the  assignment  of  contract  administration  personnel  to  service 
acquisitions  was  explored  to  discern  any  links  between  the  characteristics  of 
procurements  and  buyer  qualifications.  Reed  (2010)  discussed  some  of  the  leading 
variables  used  by  the  U.S.  Air  Force  in  the  assignment  of  contract  administration 
manpower  to  contracting  units,  which  included  the  complexity  of  acquisitions  and  the 
allowable  time  for  the  completion  of  contract  actions,  among  other  variables.  Because 
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manpower  is  later  assigned  internally  by  unit  management,  it  is  warranted  to  also 
examine  whether  the  manpower  allocation  variables — which  are  used  broadly  by  the  U.S. 
Air  Force — are  replicated  in  unit  management’s  decisions  to  internally  assign  personnel 
to  acquisitions.  Although  data  on  respondent  pay  grade  were  not  collected,  respondents 
can  be  readily  grouped  by  years  of  experience  and  by  levels  of  APDP  certification  and 
education.  Using  the  variables  provided  by  Reed  (2010)  as  a  general  basis  for  the 
assignment  of  manpower,  potential  determinants  for  the  assignment  of  administration 
personnel  to  acquisitions  are  the  contract  type,  commerciality  of  the  service,  the 
sufficiency  of  lead-time  available  for  the  procurement,  and  the  level  of  the  internal 
customer’s  commitment  to  the  acquisition,  in  which  the  level  of  the  internal  customer’s 
commitment  may  be  a  proxy  for  the  manager’s  perception  of  the  importance  of  the 
acquisition.  In  the  analysis,  linear  regression  was  used  to  examine  assignment  by 
experience — since  buyer  experience  was  represented  as  a  continuous  variable — and 
ordinal  logistic  regression  was  used  in  the  regressions  of  APDP  certification  and  buyer 
education — since  the  two  variables  are  ordered  categorical.  For  the  regression  of 
education  level  (Table  56),  buyers  were  ordered  into  the  following  four  categories:  high 
school  or  general  equivalency  diploma,  associate’s  degree,  bachelor’s  degree,  or  master’s 
degree  or  higher.  Statistically  significant  coefficients  were  found  for  the  linear  regression 
on  buyer  experience  (Table  54)  and  the  ordinal  logistic  regression  on  APDP  Level  (Table 
55).  It  cannot  be  inferred,  based  on  the  data,  that  managers  consider  buyer  education 
level  when  assigning  buyers  to  service  acquisitions.  Furthermore,  there  is  no  evidence 
that  the  level  of  commitment  by  the  internal  customer  is  a  factor  in  the  assignment  of 
buyers  to  acquisitions.  However,  lower  levels  of  buyer  experience  and  lower  buyer 
APDP  levels  were  associated  with  commercial  acquisitions,  fixed-price  acquisitions,  and 
acquisitions  with  greater  procurement  lead-time. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


103 


Variable 

Unstandardized 

coefficients 

Standardized 

coefficients 

t 

Sig-  (p ) 

B 

S.E. 

P 

(Constant) 

20.853 

3.259 

6.399 

<  0.001 

RBV 1 

-0.394 

0.143 

-0.185 

-2.750 

0.006 

ICC 

0.130 

0.173 

0.050 

0.749 

0.455 

Fixed  Price 

-3.757 

1.774 

-0.139 

-2.118 

0.035 

Commercial 

-3.844 

1.582 

-0.161 

-2.430 

0.016 

Note.  Dependent  Variable:  CBV1  (R2=  0.10) 


Table  55.  Ordinal  Logistic  Regression  of  APDP  Level  on  Assignment  Factors 


95%  C.I. 


Estimate 

S.E. 

Wald 

df 

Sig. 

Lower 

bound 

Upper 

bound 

Threshold 

[APDP  =  0] 

-4.519 

0.704 

41.176 

1.000 

<0.001 

-5.900 

-3.139 

[APDP  =  1] 

-3.532 

0.686 

26.500 

1.000 

<0.001 

-4.876 

-2.187 

[APDP  =  2] 

-1.737 

0.654 

7.049 

1.000 

0.008 

-3.019 

-0.455 

Location 

RBV1 

-0.050 

0.028 

3.192 

1.000 

0.074 

-0.106 

0.005 

ICC 

-0.037 

0.034 

1.185 

1.000 

0.276 

-0.103 

0.029 

Fixed  Price 

-0.616 

0.355 

3.010 

1.000 

0.083 

-1.312 

0.080 

Commercial 

-1.482 

0.327 

20.535 

1.000 

<0.001 

-2.123 

-0.841 

Note.  APDP  Level  III  is  reference  category;  Cox&  Snell  Pseudo  R2  =  0.16 


Table  56.  Ordinal  Logistic  Regression  of  Education  Level  on  Assignment  Factors 


95%  C.I. 


Estimate 

S.E. 

Wald 

df 

Sig. 

Lower 

bound 

Upper 

bound 

Threshold 

[EDUC  =  0] 

-2.669 

0.667 

15.999 

1.000 

<0.001 

-3.978 

-1.361 

[EDUC  =  1] 

-1.842 

0.653 

7.944 

1.000 

0.005 

-3.122 

-0.561 

[EDUC  =  2] 

0.076 

0.642 

0.014 

1.000 

0.905 

-1.181 

1.334 

Location 

RBV1 

-0.041 

0.028 

2.072 

1.000 

0.150 

-0.096 

0.015 

ICC 

0.029 

0.034 

0.738 

1.000 

0.390 

-0.037 

0.096 

Fixed  Price 

-0.404 

0.352 

1.318 

1.000 

0.251 

-1.093 

0.286 

Commercial 

-0.327 

0.312 

1.098 

1.000 

0.295 

-0.939 

0.285 

Note.  Education  Level  3  is  reference  category;  Cox&  Snell  Pseudo  R2  =  0.03 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


104 


7. 


Service  Quality  Scale 


The  scale  that  was  proposed  for  measuring  B2B  service  quality  exhibited 
excellent  consistency,  with  a  reliability  coefficient  of  0.96  after  purification  in  EFA.  The 
resultant  factor  contained  8  of  the  original  14  scale  items,  including  3  from  the  proposed 
dimension  responsiveness  to  requirements.  Loadings  from  the  latent  variable  service 
quality  to  each  of  these  3  items  in  CFA  were  high.  Changes  in  the  variance  of  the  latent 
variable  explained  83%  of  the  variation  in  the  first  item,  SQj\  85%  of  the  variation  of  the 
second  item,  SQk;  and  83%  of  the  variation  in  the  third  item,  SQm,  as  measured  by  the 
coefficient  of  determination,  R  .  Table  57  presents  measurement  statistics  for  all  items  in 
the  new  dimension. 


Table  57,  Measurement  Item  Statistics  for  Responsiveness  to  Requirements 


Item 

Mean 

St.  Dev. 

EFA 

loading 

Endogenous  measurement  model 

Standardized  ,  Residual 

Rz 

loading  estimate  variance 

SQj 

The  contractor  met  the 

5.78 

1.23 

0.901 

0.909 

0.826 

0.174 

requirements  of  the 

contract. 

SQk 

The  contractor 

5.76 

1.18 

0.898 

0.919 

0.845 

0.155 

satisfied  our  need. 

SQl 

The  contractor 

5.79 

1.23 

-  Not  used  for  measurement  - 

performed  the  work  we 

needed  it  to  do. 

SQm 

The  timeliness  of  the 

5.71 

1.25 

0.894 

0.908 

0.824 

0.175 

contractor's  work. 

SQn 

The  quality  of  the 

5.38 

1.23 

-  Not  used  for  measurement  - 

contractor's  work. 
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K.  CONCLUSION 


In  summary,  this  chapter  reported  the  results  from  tests  of  the  sample  data  using 
multiple  methods.  First,  the  240  usable  survey  responses  were  analyzed  to  verify  that 
SEM  assumptions  were  met  prior  to  testing.  EFA  was  then  performed  to  purify  scales,  in 
which  a  total  of  26  measurement  items  were  retained  for  7  constructs,  with  all  constructs 
being  measured  by  no  less  than  3  items  and  exhibiting  sufficient  reliability  coefficients. 
Construct  validity  was  then  assessed  using  CFA.  Constructs  met  the  four  rules  of  thumb 
for  construct  validity  that  were  proposed  by  Hair  et  al.  (2010),  and  an  assessment  of  fit 
indices  revealed  that  measurement  models  exhibited  good  fit  to  the  sample  data.  Finally, 
the  hypothesized  structural  model  was  estimated  using  SEM,  which  fit  well  to  the  sample 
data  but  resulted  in  several  non-significant  path  estimates.  These  paths  were  later 
trimmed  from  the  structural  model,  along  with  manifest  and  latent  variables  where 
appropriate,  resulting  in  a  second  structural  model.  This  model  also  fit  well  to  the  sample 
data  and  was  found  to  be  the  preferred  model  of  the  two,  when  competed  using  two 
popular  information  criterions.  Additional  testing  was  also  performed,  when  appropriate, 
to  further  explore  relationships.  For  ease  of  interpretation,  a  summary  table  containing 
the  significance  level  of  each  hypothesis  from  the  trimmed  model  can  be  found  in  Table 
58. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


106 


Table  58.  Summary  of  Support  for  Hypotheses  in  Trimmed  Structural  Model 


Hypotheses 


Supported? 


HI  The  greater  the  compliance  with  regulations  and  statutes,  the  greater  the  service  quality. 

H2  There  will  be  a  direct,  positive  relationship  between  the  sufficiency  of  the  requirement 
definition  and  service  quality. 

H3  There  will  be  a  direct,  positive  relationship  between  the  sufficiency  of  the  requirement 
definition  and  regulatory  and  statutory  compliance. 

H4  The  greater  the  extent  to  which  the  contractor  defines  requirements,  the  greater  the  service 
quality. 

H5  The  greater  the  extent  to  which  the  contractor  defines  requirements,  the  greater  the 
sufficiency  of  the  requirement  definition. 

H6  The  greater  the  communication  between  the  government  and  the  contractor,  the  greater  the 
service  quality. 

H7  The  greater  the  communication  between  the  government  and  the  contractor,  the  greater  the 
regulatory  and  statutory  compliance. 

H8  The  greater  the  amount  of  government  surveillance  of  contractor  performance,  the  greater  the 
service  quality. 

H9  The  greater  the  extent  to  which  performance-based  services  acquisition  procedures  are  used, 
the  greater  the  service  quality. 

H10  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater  the  service  quality. 

HI  1  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater  the  compliance  with 
regulations  and  statutes. 

H12  The  greater  the  sufficiency  of  procurement  lead-time,  the  greater  the  sufficiency  of  the 
requirement  definition. 

H13  The  greater  the  turnover  of  government  acquisition  personnel,  the  lesser  the  service  quality. 

H14  The  greater  the  turnover  of  government  acquisition  personnel,  the  lesser  the  compliance  with 
regulations  and  statutes. 

HI 5  There  will  be  a  direct,  positive  relationship  between  the  amount  of  contract  administrator 
experience  and  the  sufficiency  of  the  requirement  definition. 

HI 6  There  will  be  a  direct,  positive  relationship  between  the  amount  of  contract  administrator 
experience  and  regulatory  and  statutory  compliance. 

HI 7  The  greater  the  contract  administrator's  APDP  certification  level,  the  greater  the  sufficiency 
of  the  requirement  definition. 

HI 8  The  greater  the  contractor  administrator's  APDP  certification  level,  the  greater  the  regulatory 
and  statutory  compliance. 

HI 9  The  greater  the  contract  administrator's  education  level,  the  greater  the  sufficiency  of  the 
requirement  definition. 

H20  The  greater  the  contract  administrator's  education  level,  the  greater  the  regulatory  and 
statutory  compliance. 

H21  The  greater  the  internal  customer's  commitment  to  the  service  acquisition,  the  greater  the 
compliance  with  regulations  and  statutes. 

H22  The  greater  the  internal  customer's  commitment  to  the  service  acquisition,  the  greater  the 
sufficiency  of  the  requirement  definition. 


Noc 

Yesa 

Yesb 

No 

Noc 

Yesa 

Yesa 

No 

No 

Noc 

No 

Yesa 

Nod 

Yesb 

No 

No 

No 

No 

No 

No 

No 

Yesa 


Notes.  *p  <  0.05,  b£>  <  0.10,  cSignificant  but  inverse  estimate,  Nonsignificant  but  retained  in  trimmed  model  ( t  >  1.00). 
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V.  DISCUSSION  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

The  objective  of  this  research  was  to  bridge  a  literature  gap  and  to  offer  service 
contract  practitioners  within  the  U.S.  Air  Force  a  comprehensive  model  to  better  improve 
the  acquisition  of  services  while  increasing  compliance  with  federal,  DoD,  and  service 
acquisition  regulations.  The  study  addressed  the  following  three  research  questions: 

Research  Question  1:  What  are  the  determinants  of  services  sourcing 
performance? 

Research  Question  2:  Which  determinant(s)  have  the  greatest  impact  on  the  key 
contract  outcomes  of  service  quality  and  regulatory  and  statutory  compliance? 

Research  Question  3:  How  can  service  quality  be  validly  measured  in  a  business- 
to-business  context? 

A  new  gaps  model  was  proposed  that  extended  previous  work  by  Parasuraman  et 
al.  (1985)  and  Kong  and  Mayo  (1993)  into  the  context  of  business-to-business  (B2B) 
services  in  which  service  requirements  are  defined  by  the  customer.  This  framework  was 
also  utilized  to  create  a  service  quality  measurement  scale  for  B2B  services  in  which  two 
of  the  five  SERVQUAL  dimensions  proposed  by  Parasuraman  et  al.  (1988)  were  retained 
and  a  new  dimension,  responsiveness  to  requirements,  was  added.  The  scale  exhibited  an 
exceptionally  high  reliability  coefficient,  and  further  testing  in  exploratory  factor  analysis 
(EFA)  and  confirmatory  factor  analysis  (CFA)  provided  support  for  scale  validity. 

A  structural  model  of  the  determinants  of  services  sourcing  performance  was  also 
tested  and  found  to  exhibit  good  fit  to  the  sample  data,  which  were  collected  from  240 
U.S.  Air  Force  contract  administrators  and  analyzed  using  structural  equation  modeling 
(SEM).  The  model  was  later  trimmed  to  achieve  a  greater  degree  of  parsimony  and  to 
allow  for  a  more  precise  and  reliable  replication  in  future  research.  This  trimmed  model 
was  empirically  found  to  be  the  preferred  model  of  the  two  through  an  examination  of  the 
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trimmed  model’s  goodness-of-fit  to  the  sample  data  and  a  competition  of  both  structural 
models  using  information  criterion  measures. 

Finally,  hypothesized  causal  paths  within  the  two  structural  models  were 
estimated  using  SEM  to  test  22  hypothesized  relationships.  Further  analysis  was  also 
perfonned,  when  appropriate,  through  other  statistical  techniques  such  as  analyses  of 
variance  and  linear  and  non-linear  regression.  Many  of  the  findings  from  these 
hypothesis  tests  have  significant  implications  to  the  U.S.  Air  Force’s  and  DoD’s 
management  of  service  acquisitions.  Among  them  is  a  finding  that  the  adequacy  of 
government-contractor  communication  has  the  greatest  impact  on  the  level  of  service 
perfonnance  and  also  has  an  impact  on  the  extent  to  which  the  contract  is  compliant  with 
regulations  and  statutes.  Further  findings  provide  support  for  the  importance  of 
sufficiently  defining  service  requirements  and  for  establishing  proper  levels  of 
commitment  to  acquisitions  by  internal  customers.  An  analysis  of  acquisition  personnel 
turnover  resulted  in  two  practical  findings  in  which  statistically  significant  differences  in 
the  levels  of  contract  compliance  (with  regulation  and  statutes)  were  discovered  between 
groups  of  contracts  with  high  acquisition  personnel  turnover  and  groups  of  contracts  with 
low  acquisition  personnel  turnover.  Finally,  a  negative  association  was  found  between 
the  service  outcomes  of  compliance  with  regulations  and  statutes  and  service  quality — a 
finding  that  not  only  presents  managerial  implications  but  that  may  warrant  further 
research  to  determine  cause. 

B.  DISCUSSION  AND  IMPLICATIONS 

This  section  presents  a  discussion  of  the  research  results  as  well  as  of  the 
theoretical  and  managerial  implications  of  the  research.  The  results  from  the  SEM 
analysis  of  the  trimmed  model  are  discussed  when  path  estimates  and  their  respective 
effect  sizes  and  significance  levels  are  referenced,  as  the  trimmed  model  was  found  to  be 
the  most  preferred  model  in  the  study.  This  model  exhibited  good  fit  to  the  sample  data 
and  provided  support  for  seven  of  the  hypotheses.  Empirically,  the  SEM  analysis 
resulted  in  statistically  significant  estimates  for  eight  of  the  causal  paths  (Hypotheses  1, 
2,  3,  6,  7,  10,  12,  and  22),  moderately  significant  estimates  for  two  of  the  causal  paths 
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(Hypotheses  5  and  14),  and  estimates  with  no  statistical  support  for  12  of  the  causal  paths 
(Hypotheses  4,  8,  9,  11,  13,  and  15  through  21). 

The  order  of  the  discussion  in  this  section  will  coincide  with  the  order  of  the 
sections  within  the  literature  review  and  with  the  numerical  order  of  the  22  hypotheses. 
The  proposed  gaps  model  and  the  effectiveness  of  the  proposed  scale  for  the 
measurement  of  service  quality  in  customer-defined  B2B  service  acquisitions  will  be 
discussed  first  and  will  be  followed  by  discussions  of  the  results  from  hypothesis  testing. 

1.  Gaps  Model  and  Service  Quality  Scale 

The  primary  theoretical  implication  of  this  study  is  the  extension  of 
service  quality  theory  into  B2B  contexts.  The  previous  work  of  Kong  and  Mayo  (1993) 
attempted  to  do  this  but  fell  short  because  their  framework  was  limited  to  interfirm 
account  services  and  was  not  directly  applicable  to  B2B  purchasing;  a  flaw  in  the  model 
did  not  allow  for  generalization  to  service  acquisitions  in  which  service  specifications  are 
defined  by  the  customer  and  not  the  service  supplier. 

Furthennore,  a  scale  for  the  measurement  of  B2B  service  quality  was 
developed  based  on  adaptations  of  Pasaruraman  et  al.’s  (1988)  SERVQUAL  scale,  in 
which  two  of  the  authors’  five  service  quality  dimensions  were  retained  and  a  new 
dimension,  responsiveness  to  requirements,  was  added.  The  scale  perfonned  well  in 
EFA;  all  service  quality  measurement  items  loaded  on  one  factor,  once  purified  from  14 
to  8  items,  and  exhibited  an  exceptionally  high  scale  reliability  coefficient  (Cronbach’s 
alpha  of  0.96).  Three  of  the  eight  measurement  items  retained  after  EFA  were  from  the 
newly  proposed  dimension  and  each  presented  factor  loadings  greater  than  0.89  on  the 
primary  factor,  indicating  that  responsiveness  to  requirements  successfully  integrated 
with  the  two  SERVQUAL  dimensions  that  were  retained.  This  new  scale  offers  value  to 
researchers.  First,  it  can  be  used  to  guide  future  research  in  the  context  of  B2B  service 
acquisitions.  There  is  an  apparent  need  for  further  improvements  in  the  management  of 
services  acquisitions;  the  necessary  first  step  is  to  apply  valid  measures  when  quality  is 
examined.  Second,  existing  service  quality  scales  were  not  directly  applicable  to  B2B 
services  since  they  emphasized  the  delivery  of  off-the-shelf  services  by  suppliers  and 
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assumed  face-to-face  contact  between  customers  and  suppliers.  Previous  findings  based 
on  these  scales  may  need  to  be  revisited  since  the  measures  that  were  used  may  not  have 
been  appropriate  or  valid  in  B2B  contexts. 

2.  Regulatory  and  Statutory  Compliance 

Hypothesis  1:  The  greater  the  compliance  with  regulations  and 

statutes,  the  greater  the  service  quality. 

The  modeled  path  for  Hypothesis  1  was  found  to  be  statistically 
significant  with  a  moderate  effect  size.  However,  the  data  did  not  support  the  a  priori 
positive  path  coefficient.  Instead,  the  model  estimated  a  statistically  significant,  non¬ 
positive  coefficient  that  reflected  the  following  relationship:  the  lesser  the  compliance 
with  regulations  and  statutes,  the  greater  the  service  quality.  As  such,  service  quality  and 
compliance  with  regulations  and  statutes  are  somewhat  mutually  exclusive.  Several 
potential  explanations  of  this  result  are  offered. 

First,  it  is  plausible  that  some  of  the  regulations  and  statutes  applicable  to 
the  U.S.  Air  Force’s  acquisitions  of  services  are  successful  at  achieving  public  policy 
objectives  but  are  detrimental  to  achieving  the  expected  levels  of  service  quality.  For 
example,  competition  requirements  may  force  suppliers  in  highly  competitive  industries 
to  lower  the  quantity  or  quality  of  service  inputs  to  remain  competitive  in  proposals, 
resulting  in  lower  service  levels.  Or,  contract  administrators  who  do  not  comply  with 
competition  requirements  and,  instead,  issue  non-competitive  awards  may  be  predisposed 
to  enter  into  sole-source  negotiations  with  a  preferred  supplier.  Alternatively,  if 
competition  does  allow  for  the  selection  of  higher-performing  suppliers,  the  act  of 
compliance  with  pre-solicitation  policies,  regulations,  and  statutes  (e.g.,  synopsis 
requirements,  approvals,  and  solicitation  reviews)  may  limit  the  time  available  for 
competition.  Coopers  and  Lybrand  and  The  Analytical  Sciences  Corporation  (1994) 
found  that  compliance  with  regulations  and  statutes  drives  an  18%  cost  premium  to  the 
DoD.  It  is  plausible  that  the  true  cost  of  compliance  to  the  DoD  also  includes  a  premium 
on  performance. 
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A  second  possibility  exists  that  the  actions  necessary  to  achieve 
compliance  with  the  numerous  regulations  and  statutes  directly  reduce  the  contract 
administrator’s  ability  to  adequately  monitor  and  assess  performance  and  to  address 
deficiencies  with  the  supplier.  Compliance  with  labor  law  acts,  such  as  the  McNamara- 
O’Hara  Service  Contract  Act  or  the  Davis-Bacon  Act,  which  require  administrators  to 
perfonn  on-site  interviews  with  contractor  employees  and  to  review  payrolls,  may  unduly 
limit  the  administrator’s  available  time  to  manage  perfonnance.  As  such,  in  some 
instances,  the  government’s  level  of  contract  administration  manpower  may  also  be  low 
enough  that  personnel  are  forced  to  tradeoff  their  available  time  between  complying  with 
statues  and  regulations  and  ensuring  that  the  expected  level  of  service  performance  is 
achieved.  DoD  leadership  also  seems  to  favor  the  possibility  that  certain  policies  unduly 
overburden  the  contracting  workforce.  For  example,  the  USD(AT&L)  recently  issued 
guidance  to  reduce  non-productive  procurement  processes  through  the  elimination  of 
non-value  added  regulations  and  statutes,  among  other  steps  (USD[AT&L],  2010b).  The 
USD(AT&L)  found  that  the  DoD’s  compliance  with  internal  and  congressional 
procurement  reporting  requirements  not  only  cost  the  Department  an  estimated  $350 
million  annually  but  also  took  the  acquisition  workforce  away  from  executing  defense 
programs. 

Finally,  regulations  and  statutes  or  actions  by  the  contractor  to  comply 
with  regulations  and  statutes  may  directly  detract  from  the  contractor’s  ability  to  perfonn. 
As  before,  the  actions  necessary  for  the  contractor  to  comply  with  labor  laws  may  detract 
from  the  time  available  to  maintain  service  levels.  These  laws  may  also  prove  to  be  a 
disincentive  to  the  hiring  of  additional  labor  if  the  applicable  prevailing  wage  rates  are 
higher  than  the  wages  typically  paid  by  the  finn.  Alternatively,  the  focus  on  compliance 
may  regress  the  government-contractor  relationship  to  that  of  a  compliance  orientation 
instead  of  a  relationship  built  on  the  relational  exchange  norms. 
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3. 


Requirement  Definition 


Hypothesis  2:  There  will  be  a  direct,  positive  relationship  between  the 
sufficiency  of  the  requirement  definition  and  service  quality. 

Hypothesis  2  was  supported  in  the  model,  with  a  statistically  significant 
parameter  estimate  approaching  a  moderate  effect  size.  This  finding  supports  non- 
empirical  claims  of  other  authors  who  have  argued  that  sufficiently  defining  performance 
requirements  is  critical  to  obtaining  desired  levels  of  service.  It  also  provides  empirical 
support  for  the  proposed  gaps  model,  in  which  end-user  needs  are  translated  into  quality 
specifications  and  the  difference  between  service  expectation  and  perceptions  is 
dependent  on  the  sufficiency  of  the  requirements  documents.  Furthermore,  this  result 
bolsters  the  need  to  address  the  requirements  creep  caused  by  poorly  defined  service 
specifications,  as  recently  discussed  in  a  USD(AT&L)  guidance  memorandum 
(USD[AT&L],  2010b). 

Hypothesis  3:  There  will  be  a  direct,  positive  relationship  between 
the  sufficiency  of  the  requirement  definition  and  regulatory  and 
statutory  compliance. 

The  trimmed  model  offered  moderate  support  for  Hypothesis  3  and  the 
associated  argument  that  a  well-defined  requirement  definition  should  also  adequately 
define  those  actions  required  of  the  contractor  to  ensure  compliance  with  regulations  and 
statutes.  A  wide  variety  of  service  types,  contract  types,  and  award  amounts  were 
represented  by  survey  respondents  in  this  study.  Since  compliance-related  actions 
required  by  the  contractor  are  also  likely  to  vary  based  on  these  factors,  the  level  of 
significance  and  the  small  effect  size  obtained  from  the  SEM  analysis  are  not  surprising. 
For  example,  requirements  documents  for  a  firm-fixed-price  acquisition  of  custodial 
services  would  likely  contain  far  fewer  compliance-related  requirements  for  the  supplier 
than  a  cost-plus-award-fee  contract  for  research  and  development.  As  such,  the  proper 
estimation  of  this  path  may  require  the  inclusion  of  moderating  effects  (i.e.,  interactions) 
due  to  contract  complexity  or  other  effects.  Nonetheless,  it  can  be  concluded  that 
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sufficiently  defining  the  service  requirement  is  a  critically  important  task  because  it 
affects  both  service  quality  and  the  amount  of  compliance  with  regulations  and  statutes. 
As  such,  managers  must  make  certain  that  performance  requirements  and  compliance- 
related  requirements  are  well  defined  in  requirements  documents  to  achieve  desired 
acquisition  outcomes. 

Hypothesis  4:  The  greater  the  extent  to  which  the  contractor 

defines  requirements,  the  greater  the  service  quality. 

The  SEM  analysis  did  not  provide  support  for  Hypothesis  4.  At  first 
glance,  this  result  seems  to  indicate  that  early  integration  of  suppliers  into  the  planning 
and  solicitation  phases  of  acquisitions  has  no  direct,  significant  effect  on  service  quality, 
which  is  counter  to  the  findings  of  other  authors  who  have  argued  for  the  effectiveness  of 
early  supplier  integration  (see,  for  example,  Briscoe  et  ah,  2004).  However,  the  lack  of 
support  for  this  hypothesis  does  not  mean  that  early  integration  has  no  effect  on  service 
levels  since  the  scale  used  in  this  study  to  measure  service  quality  is  based  on  gaps 
between  expectations  and  actual  observations  of  service  levels  and  since  the  model  path 
also  included  a  mediating  variable.  Because  the  hypothesized  relationship  has  a  direct 
effect  on  service  quality  (excluding  the  indirect  effect  in  which  requirement  definition 
sufficiency  is  a  mediating  variable),  the  causal  path  only  represents  direct  effects  on 
changes  to  expectations  and/or  observations  of  service  levels;  effects  due  to  changes  in 
the  sufficiency  of  the  requirement  definition  are  accounted  for  elsewhere  in  the  model. 
Therefore,  supposing  that  the  expected  service  level  prior  to  award  is  defined  as  E,  the 
supplier  involvement  in  the  definition  of  requirements  results  in  a  change  in  expectations, 
or  A E,  and  the  government  expectation  of  the  service  level  after  pre-award  interaction 
with  the  contractor  is  E\  where  E’  =  E  +  A E.  If  the  observed  level  of  service  (O),  is 
commensurate  with  the  buyer’s  expected  level  of  service,  as  communicated  in  the 
requirements  documents,  then  O  =  E,  O’  =  E’,  and  AO  must  equal  A E.  In  reality,  it 
would  be  unreasonable  to  expect  that  the  observed  service  level  always  equals  the 
expected  service  level  ( O  =  E);  however,  it  is  reasonable  to  expect  that  a  prudent  supplier 
would  adequately  perform  the  portion  of  the  service  that  represents  the  change  in 
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expectations  due  to  the  supplier’s  recommendations.  As  such,  it  can  still  be  expected  that 
the  change  in  the  observed  service  level  equals  the  change  in  the  expected  service  level 
(AO  is  still  equal  to  A E),  and  the  estimate  of  the  causal  path  from  extent  contractor 
defined  requirements  to  service  quality  would  not  be  expected  to  differ  significantly  from 
0.  Therefore,  this  non-significant  finding  should  not  be  interpreted  to  mean  that  the  early 
integration  of  suppliers  has  no  effect  on  service  levels,  but  rather  that  any  increases  or 
decreases  in  observed  service  levels  due  to  early  integration  were  likely  offset  by  similar 
increases  or  decreases  in  expectations;  when  government-contractor  communications 
prior  to  contract  award  changed  the  government’s  expected  service  level,  the  contractor 
adequately  perfonned  to  the  change  if  the  change  in  expected  service  level  was  not 
formalized  in  the  requirements  documents. 

Hypothesis  5:  The  greater  the  extent  to  which  the  contractor 
defines  requirements,  the  greater  the  sufficiency  of  the  requirement 
definition. 

Under  ideal  conditions,  the  contractor’s  overall  level  of  service 
perfonnance  would  be  the  same  as  the  level  of  performance  specified  in  the  requirements 
documents,  and  these  documents  would  accurately  reflect  government  expectations,  such 
that  the  expected  service  level  would  equal  the  observed  service  level.  However,  in 
reality  and  in  the  sample  used  in  this  study,  it  is  atypical  for  requirements  documents  to 
fully  and  accurately  communicate  the  government’s  expectations  for  performance  of  the 
service.  As  such,  it  seems  reasonable  that  supplier  interactions  in  defining  requirements 
would  also  change  how  sufficiently  the  requirement  definition  describes  the 
government’s  service  requirements.  The  model  estimated  a  small  and  non-positive 
relationship  between  the  extent  to  which  the  contractor  defined  requirements  and  the 
sufficiency  of  the  requirement  definition,  implying  that  greater  involvement  by  the 
contractor  in  defining  service  requirements  leads  to  a  less  sufficient  definition  of  those 
requirements — perhaps  due  to  a  greater  gap  between  the  contractor’s  version  of  the  work 
and  the  government’s  true  need. 
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The  results  of  Hypothesis  4  and  Hypothesis  5  suggest  that  early 
integration  of  suppliers  into  procurement  planning  may  be  productive,  or  at  least  not 
detrimental,  in  aligning  government  expectations  with  contractor  capabilities  and  industry 
standards.  However,  the  government’s  acquisition  objectives  may  be  at  risk  when 
potential  suppliers  directly  influence  how  these  objectives  are  communicated  in  the 
solicitation  and  contract.  Additionally,  it  is  important  to  note  that  the  survey  used  a 
single-item  scale  to  measure  the  extent  to  which  the  contractor  defined  requirements. 
Because  this  extent  cannot  be  readily  observed,  a  multi-item  scale  may  have  been  more 
appropriate  for  reliable  measurement  and  would  have  allowed  the  model  to  account  for 
any  measurement  error  in  the  variables. 

4.  Relational  Exchange 

Hypothesis  6:  The  greater  the  communication  between  the  government 
and  the  contractor,  the  greater  the  service  quality. 

Hypothesis  7:  The  greater  the  communication  between  the  government 
and  the  contractor,  the  greater  the  regulatory  and  statutory  compliance. 

Hypotheses  6  and  7  were  both  found  to  be  supported  by  the  sample  data 
and  presented  the  largest  effect  sizes  on  the  key  sourcing  outcomes  of  service  quality  and 
regulatory  and  statutory  compliance.  It  is  interesting  that  an  aspect  of  relational  exchange 
— communication — has  a  greater  effect  on  key  service  outcomes  than  do  any 
transactional  processes  (e.g.,  requirements  definition),  resources  (e.g.,  time),  or 
characteristics  of  the  involved  personnel  (e.g.,  experience).  This  finding  is  in  stark 
contrast  to  the  way  acquisition  personnel  are  trained,  educated,  and  developed.  Nowhere 
in  their  development  are  these  valuable  soft  skills  taught.  Perhaps  more  time  should  be 
devoted  to  the  development  and  sustainment  of  relational  norms  such  as  solidarity, 
mutuality,  flexibility,  reciprocity,  trust,  commitment,  and  harmonization  of  conflict. 
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5.  Agency  Theory 


Hypothesis  8:  The  greater  the  amount  of  government  surveillance  of 

contractor  performance,  the  greater  the  service  quality. 

Hypothesis  8,  the  relationship  between  the  amount  of  government 
surveillance  and  service  quality,  was  not  supported  in  the  model.  It  is  possible  that  many 
of  the  service  contractors  are  capable  of  policing  their  own  performance  or  that 
government-contractor  communication  and  trust  were  sufficient  enough  to  eliminate  the 
type  of  asymmetric  information  that  might  be  obtained  through  surveillance  of 
perfonnance.  Gundlach  and  Cannon  (2009)  also  arrived  at  a  similar  conclusion  that 
surveillance  does  not  impact  quality;  they  determined  that  information  exchange  was  the 
most  effective  verification  strategy  in  medium-  and  high-trust  relationships  and  that 
corroboration  with  external  sources  may  be  the  most  effective  strategy  in  low-trust 
relationships.  Such  a  finding  as  this  may  indicate  that  the  continuous  calls  for  the  DoD  to 
improve  its  oversight  of  contractors,  such  as  those  calls  by  the  GAO,  may  be 
unwarranted,  at  least  within  the  U.S.  Rather  than  investing  contract  management 
resources  into  increased  supplier  monitoring,  contract  administrators  may  be  better 
served  by  increasing  communications.  As  such,  relational  exchange  seems  to  better 
account  for  supplier  performance  than  do  the  precepts  of  transaction  cost  economics  or 
agency  theory. 


Hypothesis  9:  The  greater  the  extent  to  which  performance-based  services 
acquisition  procedures  are  used,  the  greater  the  service  quality. 

Hypothesis  9  was  also  found  to  be  non-significant,  indicating  that  no 
significant,  direct  relationship  exists  between  the  extent  that  PBSA  procedures  are  used 
and  service  quality.  This  is  especially  surprising  considering  that  the  benefits  from  using 
outcome-oriented  requirements  definitions  had  been  well-supported  by  literature. 
However,  the  survey  used  a  single-item  scale  to  measure  the  extent  that  PBSA  procedures 
were  integrated  into  the  acquisition.  Because  this  extent  cannot  be  readily  observed,  a 
multi-item  scale  may  have  been  more  appropriate  for  reliable  measurement  and  to 
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account  for  measurement  error.  Similarly,  the  subjective  assessments  by  respondents  as 
to  the  extent  that  PBSA  procedures  were  used  may  have  varied  greatly  with  respect  to 
interpretations  of  what  constitutes  a  PBSA  contract.  More  than  78%  of  respondents 
reported  that  the  extent  that  PBSA  procedures  were  used  was  5  or  greater  on  a  7-point 
scale,  with  a  mean  reported  value  of  5.6.  As  such,  a  subjective  measure  may  not  have 
been  effective  if,  in  reality,  the  majority  of  service  acquisitions  were  not  highly 
performance  based. 

6.  Resource-Based  View  of  the  Firm 

Hypothesis  10:  The  greater  the  sufficiency  of  procurement  lead-time,  the 
greater  the  service  quality. 

Hypothesis  11:  The  greater  the  sufficiency  of  procurement  lead-time,  the 
greater  the  compliance  with  regulations  and  statutes. 

Hypothesis  12:  The  greater  the  sufficiency  of  procurement  lead-time,  the 
greater  the  sufficiency  of  the  requirement  definition. 

Of  Hypotheses  10,  11,  and  12,  only  Hypothesis  12  was  supported  in  the 
model  as  hypothesized;  the  analysis  produced  a  moderately-sized  effect  for  this 
hypothesized  path.  The  model  estimated  a  small  and  negative  path  coefficient  for 
Hypothesis  10  that  was  statistically  significant,  supporting  an  alternative  argument  that 
service  quality  decreases  as  the  sufficiency  of  procurement  lead-time  increases.  No 
support  was  provided  in  the  model  for  Hypothesis  1 1 . 

Prior  to  considering  the  potential  causes  of  the  negative  relationship  that 
was  estimated  between  procurement  lead-time  sufficiency  and  service  quality,  it  is  most 
appropriate  to  focus  away  from  factors  that  are  explained  elsewhere  in  the  model.  For 
example,  in  the  context  of  DoD  service  acquisitions,  one  might  normally  expect  that 
quality  would  increase  when  lead-time  increases  since  the  buyer(s)  would  have  more  time 
to  perfonn  critical  pre-award  tasks  (such  as  perfonning  market  research,  ensuring  the 
adequacy  of  the  SOW/PWS,  completing  contract  reviews,  issuing  a  synopsis  and 
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solicitation,  and  evaluating  offers).  However,  most  of  these  factors  were  already 
accounted  for  elsewhere  in  the  model.  For  instance,  requirement  definition  sufficiency 
was  found  to  be  positively  related  with  the  procurement  lead-time  sufficiency  in 
Hypothesis  12.  In  fact,  a  positive,  indirect  effect  from  procurement  lead-time  sufficiency 
to  service  quality  exists  through  requirement  definition  sufficiency  as  a  mediating 
variable. 

However,  the  path  estimate  from  procurement  lead-time  sufficiency  to 
regulatory  and  statutory  compliance  was  not  significant,  indicating  that  buyers  may  be 
resilient  against  insufficient  procurement  lead-times  when  it  comes  to  compliance  with 
regulations  and  statutes.  As  such,  the  negative  path  estimate  from  Hypothesis  10  is  a 
result  of  factors  that  are  external  to  regulatory  and  statutory  compliance,  requirement 
definition  sufficiency,  and  impacts  on  service  quality  that  are  due  to  these  regulations  and 
statutes  or  to  the  sufficiency  of  the  requirement  definition. 

One  such  external  factor  was  examined  in  this  study  using  regression 
analysis,  and  it  was  found  that  the  greater  the  sufficiency  of  the  procurement  lead-time, 
the  lesser  the  experience  and  APDP  level  of  the  buyer  assigned  to  the  acquisition.  To 
further  explore  the  impacts  on  such  an  assignment  of  buyers,  it  is  helpful  to  first  step 
back  and  view  how  pre-award  contract  actions  that  occur  during  the  period  designated  as 
procurement  lead-time  can  affect  perfonnance  outcomes. 

The  measurement  of  service  quality  is  based  on  differences  between  the 
expectations  and  perceptions  of  service  levels  over  the  life  of  the  contract — after  the 
contract  has  been  awarded  and  performance  has  begun.  By  definition,  the  sufficiency  of 
procurement  lead-time  has  immediate  and  direct  impacts  on  actions  that  occur  during  pre¬ 
award  stages  of  the  acquisition.  After  contract  award,  it  is  possible  to  correct  some,  but 
not  all,  pre-award  mistakes  that  affect  service  delivery  through  the  use  of  contract 
modifications.  As  an  example,  if  the  service  requested  was  sufficiently  defined  by  the 
requirements  documents  but  did  not  meet  actual  demand,  as  was  the  case  with  the 
Department  of  Homeland  Security’s  acquisition  of  cruise  ship  berthing  during  Hurricane 
Katrina  (see,  for  example,  Snider  &  Rendon,  2008),  then  sourcing  outcomes  could  be 
negatively  affected  but  could  also  be  corrected  through  contract  modification.  Other 
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performance  issues  that  are  caused  prior  to  award,  such  as  when  supplier  evaluations 
result  in  an  award  to  a  low-performing  supplier  (i.e.,  adverse  selection),  may  not  be  as 
easily  corrected.  In  this  scenario,  evaluation  criteria  in  the  solicitation  may  have  been 
sufficient  enough  to  meet  regulatory  requirements  but  not  sufficient  enough  to  select  the 
best  supplier,  potentially  due  to  the  assignment  of  an  inexperienced  buyer.  Additional 
pre-award  errors  that  might  contribute  to  decreased  service  quality  throughout  the  life  of 
an  acquisition  are  the  selection  of  the  wrong  contract  type  or  instrument,  the 
misapplication  of  contract  incentives,  and  the  misuse  of  certain  procedures,  such  as 
commercial-item  acquisition  procedures  or  simplified  acquisition  procedures.  The 
misuse  of  these  procedures  could  prove  detrimental  to  ensuring  the  government’s  interest 
post-award  because  of  limitations  on  the  use  of  contract  clauses,  limitations  on  requiring 
cost  or  pricing  data,  or  limitations  on  the  government’s  ability  to  oversee  contractor 
perfonnance.  If  errors  such  as  these  are  made  by  inexperienced  buyers,  long-term 
acquisition  outcomes  would  likely  be  affected.  This  also  seems  to  be  supported  by 
regression  results  in  this  study  in  which  acquisitions  that  were  administered  by  personnel 
with  five  or  less  years  of  experienced  were  found  to  have  lower  levels  of  service  quality 
than  acquisitions  administered  by  more  experienced  personnel. 

Overall,  the  SEM  results  show  that  procurement  lead-time  improves  the 
sufficiency  of  the  requirement  definition,  which,  in  turn,  improves  service  quality  and 
compliance  with  regulations  and  statutes.  However,  it  seems  that  requirements  with 
sufficient  lead-time  (i.e.,  little  pressure  to  award  a  contract)  are  not  viewed  to  be  as 
important  to  the  internal  customer  or  to  contracting  personnel;  thus,  perhaps  less  attention 
is  afforded  to  the  requirement,  and  the  buyer’s  source  selection  process  is  also  less 
rigorous,  directly  decreasing  service  quality. 

Hypothesis  13:  The  greater  the  turnover  of  government  acquisition 
personnel,  the  lesser  the  service  quality. 

Hypothesis  14:  The  greater  the  turnover  of  government  acquisition 
personnel,  the  lesser  the  compliance  with  regulations  and  statutes. 
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Neither  Hypothesis  13  nor  Hypothesis  14  was  fully  supported  in  the  SEM 
model.  However,  two  highly  significant  regression  results  provided  support  for 
Hypothesis  14. 

In  the  first  supporting  regression,  turnover  was  calculated  as  a  ratio 
(percent)  and  modified  to  a  binary  variable  about  the  median  of  1.00  (100%).  A  small, 
but  highly  significant,  coefficient  was  estimated,  indicating  that  compliance  with 
regulations  and  statutes  was  significantly  less  on  services  acquisitions  in  which,  over  the 
life  of  the  contract,  more  personnel  had  turned  over  than  were  assigned.  Such  a  result  has 
strong  implications  for  managers  of  contracting  organizations  within  which  it  appears 
likely  that  the  continuity  of  tasks  necessary  for  compliance  breaks  down  as  turnover 
approaches  100%.  Furthermore,  survey  responses  indicated  that  personnel  turnover  is 
not  adequately  managed  and,  in  many  cases,  is  highly  excessive  (possibly  due  to 
deployments,  reassignments,  cannibalization  of  skilled  personnel  by  other  agencies  that 
have  recently  increased  hiring,  retirements  due  to  an  aged  workforce,  and  normal 
attrition).  In  the  sample  data,  the  mean  respondent  was  assigned  to  a  contract  in  which 
turnover  was  in  excess  of  120%.  Nearly  25%  of  respondents  were  assigned  contracts  in 
which  turnover  was  at  least  200%  and  7.5%  of  respondents  were  assigned  to  contracts  in 
which  turnover  was  at  least  300%.  Alternatively,  only  27.5%  of  respondents  were 
assigned  to  contracts  in  which  turnover  was  less  than  50%,  and  less  than  16%  of 
respondents  were  assigned  to  contracts  with  less  than  25%  turnover.  Figure  12  presents  a 
histogram  of  percent  turnover  from  the  sample. 
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Figure  12.  Histogram  of  Percent  Turnover  of  Acquisition  Personnel 

This  result  was  further  developed  when  the  length  of  time  after  the 
contract  was  awarded  was  also  considered  and  percent  turnover  was  recalculated  as  an 
annualized  percentage.  Once  again,  the  measure  was  modified  into  a  binary  variable  with 
a  cut  about  the  median,  where  median  annualized  turnover  was  42%.  Similar  to  the 
previous  regression  result,  this  relationship  from  turnover  to  compliance  with  regulations 
and  statutes  was  found  to  be  highly  significant,  approaching  a  moderate  effect  size.  The 
results  indicate  that  service  acquisitions  with  more  than  42%  annual  turnover  are 
significantly  less  compliant  with  regulations  and  statutes  than  those  service  acquisitions 
with  less  than  42%  annual  turnover.  As  before,  the  rates  of  turnover  are  alarming.  The 
mean  respondent  sampled  was  assigned  to  a  contract  in  which  personnel  had  turned  over 
in  excess  of  65%  annually.  Nearly  19%  of  respondents  were  assigned  to  contracts  in 
which  acquisition  personnel  turned  over  at  least  once  annually,  and  less  than  16%  were 
assigned  to  contracts  in  which  personnel  turned  over  less  than  10%  annually.  Figure  13 
displays  the  sampling  results  as  a  histogram. 
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Figure  13.  Histogram  of  Annualized  Percent  Turnover  of  Acquisition  Personnel 

The  results  of  testing  Hypothesis  14  and  the  demographics  of  the  data 
collected  paint  a  dismal  picture  of  the  state  of  personnel  turnover  in  the  acquisition 
workforce.  To  properly  achieve  acquisition  objectives,  specifically  that  of  compliance 
with  acquisition  regulations  and  statutes,  managers  must  drastically  reduce  the  current 
levels  of  turnover.  First,  managers  should  avoid  assigning  temporary  employees  or  those 
employees  due  to  rotate,  deploy,  separate,  or  retire  to  manage  service  contracts  and 
instead  should  assign  those  personnel  who  are  expected  to  be  retained  throughout  the  life 
of  the  contract.  Furthermore,  agencies  should  consider  establishing  goals  and  procedures 
to  reduce  personnel  turnover  and  should  direct  that  units  report  turnover  metrics 
periodically.  These  metrics  should  be  used  to  track  agency-level  trends  and  to  identify 
and  stabilize  those  units  that  exhibit  levels  of  personnel  turnover  greater  than  100%  over 
the  life  of  service  contracts,  or  42%  annually. 
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7.  Competence-Based  View  of  the  Firm 


Hypothesis  15:  There  will  be  a  direct ,  positive  relationship  between  the 
amount  of  contract  administrator  experience  and  the  sufficiency  of  the 
requirement  definition. 

Hypothesis  16:  There  will  be  a  direct,  positive  relationship  between  the 
amount  of  contract  administrator  experience  and  regulatory  and  statutory 
compliance. 

Hypothesis  17:  The  greater  the  contract  administrator’s  APDP 

certification  level,  the  greater  the  sufficiency  of  the  requirement  definition. 

Hypothesis  18:  The  greater  the  contract  administrator’s  APDP 

certification  level,  the  greater  the  regulatory  and  statutory  compliance. 

Hypothesis  19:  The  greater  the  contract  administrator’s  education  level, 
the  greater  the  sufficiency  of  the  requirement  definition. 

Hypothesis  20:  The  greater  the  contract  administrator’s  education  level, 
the  greater  the  regulatory  and  statutory  compliance. 

None  of  these  six  hypotheses  stemming  from  theory  on  the  competence- 
based  view  of  the  firm  were  found  to  be  statistically  significant  in  the  SEM  model. 
However,  several  plausible  explanations  may  exist  for  why  this  is  the  case. 

First,  it  may  be  that  the  government  contract  administrator’s  interaction  in 
the  development  of  requirements  documents  is  not  significant  enough  to  greatly  impact 
the  sufficiency  of  requirements  documents  or  the  customer’s  ability  to  communicate 
service  expectations.  In  practice,  contract  administrators  are  likely  to  perform  reviews  of 
requirements  documents  to  ensure  adequacy  (FAR,  2005),  but  administrators  are  unlikely 
to  have  an  in-depth  understanding  of  how  services  should  be  perfonned  or  the  level  of 
perfonnance  that  internal  customers  require.  This  may  be  especially  true  when 
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acquisitions  are  for  highly  technical  services  such  as  medical  services,  research  and 
development,  or  information  technology  services. 

Second,  it  is  important  to  note  that  less  experienced  personnel,  who  will 
have  a  lower  APDP  certification  level  and  are  less  likely  to  have  attained  higher  levels  of 
education,  may  be  assigned  to  administer  simpler  service  contracts  that  are  less  burdened 
with  regulations,  such  as  those  acquired  using  simplified  acquisition  procedures. 
Achieving  full  compliance  on  such  an  acquisition  may  be  more  feasible  for  an 
inexperienced  buyer  than  doing  so  on  a  large,  complex  service  acquisition  that  may 
employ  cost-type  line-items  or  undefinitized  contractual  actions.  This  theory  may  be 
substantiated,  to  some  degree,  by  the  finding  that  a  statistically  significant  difference  in 
means  of  requirement  definition  sufficiency  existed  between  APDP  Level  II  and  APDP 
Level  III  personnel,  in  which  the  average  Level  II  respondent  reported  a  higher  level  of 
requirement  definition  sufficiency.  This  finding  indicates  that  more  qualified  personnel 
may  typically  be  assigned  to  more  complex  acquisitions  in  which  contract  outcomes  and 
perfonnance  measures,  such  as  those  that  would  be  communicated  in  requirements 
documents,  are  harder  to  define.  Applying  this  relationship — that  personnel  who  possess 
the  least  experience  and  knowledge  in  the  area  of  contract  compliance  are  assigned  to  the 
simplest  service  contracts  and  those  personnel  with  the  greatest  experience  and 
knowledge  are  assigned  to  the  most  complex  service  contracts — all  personnel  may  be 
challenged  equally. 

Third,  it  should  be  considered  that  multiple  contract  administration 
personnel  are  often  assigned  to  services  acquisitions,  even  on  common  acquisitions  such 
as  base  support  services.  For  situations  in  which  multiple  administrators  are  assigned,  the 
measurement  of  experience  is  only  an  indicator  of  the  respondent’s  experience  and  not  an 
adequate  representation  of  experience  for  all  contract  administration  personnel  assigned 
to  the  service  acquisition. 

Finally,  it  is  quite  possible  that  the  contract  administrator  who  responded 
to  the  survey  is  not  the  same  individual  who  awarded  the  contract,  as  evidenced  by  the 
high  levels  of  turnover  that  were  previously  discussed.  As  such,  the  effects  from  buyer 
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experience,  training,  and  education  at  all  stages  in  the  procurement  may  not  be  properly 
accounted  for  in  the  sample  data. 

8.  Internal  Customer  Commitment 

Hypothesis  21:  The  greater  the  internal  customer’s  commitment  to  the 
service  acquisition,  the  greater  the  compliance  with  regulations  and 
statutes. 

No  support  was  provided  for  a  direct  effect  from  internal  customer 
commitment  to  regulatory  and  statutory  compliance,  although  a  significant  indirect  effect 
was  found  through  the  mediating  variable  requirement  definition  sufficiency.  The 
absence  of  a  direct  effect  is  not  surprising  when  consideration  is  given  to  the  types  of 
regulations  and  statutes  that  are  applicable  to  the  acquisition  of  services;  many  are  central 
to  contracting  as  a  function  and  require  compliance  by  the  contracting  office,  or  at  least 
require  significant  interaction  by  contracting  personnel.  As  such,  the  contract 
administrator  may  often  become  the  overseer  of  regulatory  and  statutory  requirements 
while  the  internal  customer  focuses  on  contractor  surveillance  and  issues  that  may  arise 
during  service  delivery,  at  least  in  relatively  small  service  contracts  for  operational 
support. 


Hypothesis  22:  The  greater  the  internal  customer’s  commitment  to  the 
service  acquisition,  the  greater  the  sufficiency  of  the  requirement 
definition. 

The  model  did  support  Hypothesis  22,  estimating  a  moderately  sized 
effect  that  greater  commitment  to  the  service  acquisition  by  the  internal  customer  results 
in  a  greater  sufficiency  of  the  requirement  definition.  A  review  of  the  results  from  the 
trimmed  structural  model  reveals  that  the  level  of  internal  customer  commitment  is,  in 
fact,  the  largest  determinant  of  the  sufficiency  of  the  requirement  definition.  Because  the 
requirement  definition  impacts  both  service  quality  and  regulatory  and  statutory 
compliance,  the  adequacy  of  commitment  from  the  internal  customer  also  has  second- 
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order  effects  on  service  outcomes.  Thus,  managers  should  assess  commitment  and  assign 
only  committed  employees  to  manage  service  contracts. 

C.  ANSWERS  TO  RESEARCH  QUESTIONS 

Research  Question  1:  What  are  the  determinants  of  services  sourcing 
performance? 

In  the  final  structural  model,  service  quality  was  directly  dependent  on  five 
variables:  compliance  with  statutes  and  regulations,  communication  between  the 
government  and  contractor,  sufficiency  of  the  requirement  definition,  sufficiency  of 
procurement  lead-time,  and  the  amount  of  turnover  of  acquisition  personnel.  The  other 
sourcing  performance  outcome,  compliance  with  regulations  and  statutes,  was  directly 
dependent  on  three  variables:  communication  between  the  government  and  contractor, 
the  sufficiency  of  the  requirement  definition,  and  the  amount  of  turnover  of  acquisition 
personnel.  Additional  analysis  also  uncovered  a  significant  relationship  between  the 
contract  manager’s  experience  (years)  and  the  quality  of  the  service. 

Research  Question  2:  Which  determinant(s)  have  the  greatest  impact  on  the  key 
contract  outcomes  of  service  quality  and  regulatory  and  statutory  compliance? 

The  extent  of  communication  between  the  government  and  the  contractor  appears 
to  have  the  greatest  impact  on  both  outcomes,  exhibiting  highly  significant  path  estimates 
with  large  effect  sizes  to  these  outcome  variables.  The  amount  of  compliance  with 
regulations  and  statutes  exhibited  a  significant  but  negative  and  moderately  sized  effect 
on  service  quality.  Effect  sizes  for  all  other  path  estimates  leading  to  the  contract 
outcomes  were  small  and  varied  in  the  level  of  significance;  notable  second-order  effects 
came  from  the  sufficiency  of  procurement  lead-time  and  the  level  of  commitment  to  the 
acquisition  by  the  internal  customer. 
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Research  Question  3:  How  can  service  quality  be  validly  measured  in  a  business- 

to-business  context? 

A  14-item  scale  was  proposed  in  Chapter  III  for  the  measurement  of  service 
quality  in  business-to-business  applications  that  was  found  to  exhibit  excellent  reliability, 
as  measured  by  Cronbach’s  coefficient  alpha.  This  scale  was  based  on  a  five-dimension 
SERVQUAL  scale  that  was  developed  by  Parasuraman  et  al.  (1988),  in  which  two  of  the 
original  dimensions  were  retained  and  a  new  dimension,  responsiveness  to  requirements, 
was  added.  This  new  dimension  is  comprised  of  five  items  and  addresses  a  literary  gap 
in  the  measurement  of  service  quality  in  contractual  and  business  relationships  in  which 
service  requirements  and  performance  standards  are  established  by  the  purchasing 
organization  and  not  the  service  supplier. 

D.  RECOMMENDATIONS 

Recommendation  1:  Limit  the  turnover  of  acquisition  personnel  to  no  more  than 

100%  over  the  life  of  contracts  and  42%  annually. 

Increased  levels  of  turnover  were  found  to  be  detrimental  to  the  contracting 
officer’s  ability  to  ensure  compliance  with  acquisition  regulations  and  statutes.  In 
particular,  the  results  from  regression  analysis  indicate  that  acquisitions  are  substantially 
less  compliant  with  regulations  and  statutes  once  turnover  reaches  critical  thresholds  of 
100%  over  the  life  of  the  contract,  or  42%  annually.  At  or  above  these  levels,  it  seems 
that  continuity  is  lost  in  the  contract  administration  process  and  personnel  are  forced  to 
focus  resources  away  from  compliance  and  toward  other  elements  of  administration  in 
order  to  ensure  an  adequate  level  of  performance.  Although  a  certain  level  of  personnel 
turnover  may  be  unavoidable  due  to  deployments,  retirements,  and  regular  rotations, 
managers  should  avoid  assigning  temporary  personnel  to  manage  service  acquisitions 
and,  instead,  should  assign  those  personnel  who  the  manager  best  anticipates  retaining 
throughout  the  life  of  the  contract. 

Furthennore,  agencies  should  develop  metrics  to  track  the  rates  of  acquisition 
personnel  turnover  according  to  contract  type.  Table  59  presents  an  example  of  several 
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appropriate  metrics  that  have  been  completed  using  the  data  acquired  in  this  study. 
Turnover  metrics  should  not  only  be  used  by  agencies  to  identify  turnover-related  risks  in 
classes  of  services  but  also  should  be  reported  to  the  Office  of  the  Secretary  of  Defense 
(OSD)  so  that  DoD-wide  trends  and  areas  for  improvement  can  be  identified. 
Additionally,  data  gathered  by  agencies  and  the  OSD  should  be  used  to  concentrate 
contract  reviews  and  inspections  on  classes  of  services  acquisitions  that  are  at  the  highest 
risk  of  being  non-compliant  with  acquisition  regulations  and  statutes  due  to  excessive 
levels  of  personnel  turnover. 


Table  59.  Sample  Turnover  Reporting  Metrics 


Service  Type 

Mean 

turnover 

Mean 

annualize  d 

turnover 

Mean  dollars 
per  acquisition 
personnel 

Mean  acquisition 
personnel  per 
dollar 

Maintenance/repair 

120.19% 

57.14% 

$4,946,596.58 

1.81E-05 

Professional,  administrative 
and  management  support 

103.54% 

66.56% 

$5,211,690.23 

8.68E-06 

Utilities  and  housekeeping 

118.42% 

62.06% 

$3,036,980.78 

1.02E-05 

Medical 

98.87% 

99.08% 

$2,628,672.47 

1.77E-05 

Education  and  training 

195.89% 

87.23% 

$3,597,922.50 

4. 88E-06 

Architect-engineering 

101.56% 

65.31% 

$2,351,853.94 

9.48E-06 

Quality  control,  testing,  and 
inspection 

75.00% 

45.94% 

$719,448.75 

4.94E-06 

Research  and  development 

45.83% 

12.83% 

$1,354,166.67 

1.31E-06 

Other 

136.29% 

72.35% 

$7,203,481.92 

6.59E-06 

Recommendation  2:  Establish  buyer-supplier  relationship  management  skills  as  a 
core  competency  of  the  acquisition  workforce  through  the  incorporation  of  skills 
training  into  acquisition  curriculums  at  all  levels. 

The  extent  of  communication  between  the  government  and  contractor  was  found 
to  have  the  greatest  impact  on  contract  outcomes.  As  Gundlach  and  Cannon  (2009) 
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asserted,  it  is  reasonable  that  information  exchanges  and  the  development  of  relational 
norms,  particularly  of  trust  between  the  contractual  parties,  enhance  performance  and 
may  be  more  appropriate  methods  of  verification  than  surveillance  and  formal  evaluation, 
at  least  in  the  case  of  some  procurements  in  which  low  levels  of  trust  are  not  present. 
DoD  acquisition  training  programs  do  not  currently  focus  on  the  development  of  buyer- 
supplier  relationship  management  skills.  For  example,  nowhere  does  the  Defense 
Acquisition  University  or  on-the-job  training  teach  the  strategic  segmentation  of  spend 
(see,  for  example,  Kraljic,  1983).  This  is  a  prerequisite  for  not  only  sourcing  strategy  but 
also  for  supplier  management  because  it  determines  the  appropriate  relationship — 
transaction  or  collaborative.  In  reality,  many  strategic  suppliers  are  treated  as 
transactional,  and  far  too  many  resources  are  utilized  on  other  spend  that  is  truly 
transactional  (e.g.,  the  procurement  of  furniture).  Acquisition  training  programs  should 
capitalize  on  opportunities  to  increase  buyer-supplier  relationship  management  skills 
such  as  communication;  acquisition  personnel  need  to  know  how  to  effectively 
communicate  expectations  for  perfonnance,  keep  all  parties  infonned  of  new 
developments,  provide  infonnation  that  helps  both  parties  achieve  acquisition  objectives, 
effectively  communicate  the  government’s  needs,  and  listen  carefully  to  communications 
from  the  contractor  and  other  acquisition  team  members. 

Recommendation  3:  Ensure  that  assigned  internal  customers  are  fully  committed 

to  services  acquisitions  prior  to  engaging  in  procurement  processes. 

The  results  of  the  SEM  analysis  in  this  study  indicate  that  a  high  level  of 
commitment  from  the  internal  customer  is  necessary  to  sufficiently  define  acquisition 
requirements  and  ultimately  procure  high-quality  services.  The  internal  customers  of 
service  acquisitions  need  to  devote  the  manpower  necessary  to  adequately  communicate 
requirements  to  prospective  contractors;  to  develop  evaluation  criteria  and  participate  in 
source  selections;  to  assess  and  manage  contractor  performance  through  surveillance, 
information  exchanges,  or  other  verification  methods;  to  manage  changes  to  service 
requirements;  and  to  communicate  with  suppliers  during  performance.  Furthermore, 
internal  customers  must  commit  to  assigning  their  best  talent  to  managing  services 
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contracts,  as  Ellram  et  al.  (2007)  asserted.  Specifically,  managers  should  identify  and 
assign  those  personnel  who  exhibit  traits  similar  to  those  measured  in  the  internal 
customer  commitment  scale:  pride  in  membership  of  the  service  acquisition  team,  a  sense 
of  purpose  when  acquisition  objectives  are  achieved,  care  for  the  long-term  success  of  the 
acquisition,  and  dedication  to  ensuring  the  acquisition  meets  requirements.  Although 
these  traits  are  subjective  in  nature,  feedback  from  supervisors  and  all  members  of  the 
acquisition  team  may  allow  for  the  identification  of  the  most  suitable  personnel  as  well  as 
for  the  identification  of  those  personnel  who  may  be  less  qualified,  or  less  appropriate, 
for  an  assignment  to  managing  a  service  acquisition. 

Furthennore,  the  manpower  standards  of  internal  customers  may  need  to  be 
reviewed  to  detennine  whether  units  are  being  adequately  staffed  to  handle  service 
contracts.  For  example,  many  of  the  U.S.  Air  Force’s  common  installation-level  service 
contract  requirements  are  generated  by  civil  engineering  (CE)  units  and  are  managed,  in 
part,  by  the  CE  operations  flight.  The  Air  Force  Manpower  Standard  (AFMS)  44EO  is 
used  to  determine  the  appropriate  level  of  manning  for  this  flight,  in  conjunction  with  Air 
Force  Instruction  63-124  (U.S.  Air  Force,  2005),  but  does  not  specify  how  units  should 
assign  personnel  for  the  management  of  service  acquisitions  or  the  appropriate  level  of 
manning  for  acquisition-related  tasks  such  as  perfonning  quality  assurance  functions 
(U.S.  Air  Force,  2000).  Organizations  such  as  CE  that  lack  a  methodology  for 
determining  manning  levels  for  service  contract  management  may  consider  adopting 
some  of  the  standards  used  by  the  U.S.  Air  Force  contracting  community,  which 
considers  acquisition  type  and  value,  among  other  factors,  when  detennining  adequate 
unit  manning  (U.S.  Air  Force,  2001). 

Recommendation  4:  Pursue  initiatives  to  better  educate  customers  on  methods  to 
effectively  develop  requirements  documents;  develop  high-quality  requirements 
documents  templates  for  commonly  acquired  services. 

The  government’s  ability  to  write  sufficiently  defined  requirements  is  paramount 
to  achieving  service  levels  that  meet  customer  expectations.  However,  few  enterprise- 
level  training  programs  exist  that  instruct  customers  how  to  best  define  their 
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organization’s  requirements.  Similarly,  no  APDP  certification  standard  is  established  for 
such  personnel,  who  may  have  few  or  no  resources  to  rely  on  when  tasked  with  the 
planning  of  service  procurements.  Instituting  certification  programs  for  internal 
customers  will  not  only  increase  the  sufficiency  of  requirement  definitions  but  also  may 
increase  personnel  efficiency,  increase  procurement  lead-times,  and  decrease  post-award 
modifications  to  customer  requirements.  Similarly,  agencies  should  mandate  monthly 
contracting  stand-down  days  for  contract  managers,  contracting  officers,  internal 
customers,  and  program  managers.  In  these  sessions,  expert  trainers  should  provide  live 
continuous  learning  over  the  Internet  via  a  centralized  training  platform  and  should 
provide  personnel  with  updates  on  best  practices,  acquisition  tools,  and  policy  changes. 

Furthennore,  offering  high-quality  templates  of  requirements  documents  for  the 
most  commonly  acquired  services  may  increase  the  quality  of  those  services,  promote 
greater  compliance  with  regulations  and  statutes  such  as  the  acquisition  of  commercial 
services  and  the  use  of  PBSA  procedures,  facilitate  strategic  sourcing,  increase  lead- 
times,  and  increase  the  availability  of  internal  customer  manpower  to  prepare  for  source 
selection  and  post-award  functions.  As  such,  agencies  should  consider  establishing  an 
electronic  repository  of  work  statements  for  commonly  acquired  commercial  services, 
such  as  custodial  services  and  refuse  services.  Such  a  system  would  allow  for  the  sharing 
of  high-quality  work  statements  and  would  permit  agencies  to  track  and  report  the 
number  of  different  requirements  documents  that  are  in  use  for  each  commercial  service. 
Agencies  should  also  establish  goals  for  the  standardization  of  work  statements  for 
certain  acquisitions  and  should  use  metrics  from  the  repository  to  track  status  by 
organization. 

Finally,  agencies  should  consider  requiring  that  units  provide  monthly  reports  of 
the  number  of  contract  changes  and  contractor  claims  that  result  from  poor  specifications 
and  insufficiently  defined  requirements.  These  metrics  should  be  included  in  the  criteria 
for  unit  awards  and  be  a  mandatory  input  to  annual  performance  reports  of  unit  directors 
and  commanders  of  contracting  units  as  well  as  to  those  of  internal  customers. 
Furthermore,  these  metrics  should  be  used  to  identify  trends  in  the  sufficiency  of 
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requirements  documents  for  commonly  procured  services  and  to  evaluate  improvements 
gained  by  the  standardization  of  work  statements. 

Recommendation  5:  Revise  or  remove  non-value  added  regulations  and  statutes; 
investigate  which  regulations  and  statutes  are  detrimental  to  acquisition 
outcomes. 

A  non-positive  relationship  was  found  between  the  extent  of  compliance  with 
regulations  and  statutes  and  service  quality.  This  result  suggests  that  the  effects  of 
regulations  and  statutes  should  be  analyzed  to  assess  whether  any  are  directly  detrimental 
to  service  quality  or  whether  the  individual  act  of  compliance  with  a  regulation  or  statute 
or  the  collective  act  of  compliance  with  all  overburdens  contract  administrators  to  the 
point  that  resources  are  insufficiently  allocated  to  the  management  of  contractor 
performance.  Additional  analysis  should  also  be  performed  at  the  unit  level  to  ensure 
that  compliance  is  equal  among  all  contracts  and  that  contracts  with  high-performing 
suppliers  are  not  overlooked  or  fded  away. 

Recommendation  6:  Implement  a  performance-evaluation  system  for  service 
contracts  using  the  service  quality  scale. 

Agencies  should  integrate  the  service  quality  scale  into  a  supplier  performance- 
evaluation  system  to  assess  and  rate  the  performance  of  service  suppliers  throughout  the 
life  of  acquisitions.  Simpson,  Siguaw,  and  White  (2002)  argued  for  the  importance  of 
regular  feedback  between  buyers  and  suppliers.  With  regular  and  continuous  feedback, 
suppliers  can  concentrate  resources  on  improving  those  dimensions  of  service  delivery 
that  are  most  important  to  the  buyer.  However,  the  DoD’s  current  method  of  rating 
contractor  performance,  the  contractor  performance  assessment  reporting  system 
(CPARS),  with  few  exceptions,  only  assesses  performance  annually  and  is  more 
appropriate  for  use  as  past  performance  information  (i.e.,  used  to  inform  source  selection 
decisions)  than  as  a  means  of  continuous  supplier  improvement  (Straight,  1999). 
Periodic  ratings  using  a  service  quality  scale  could  address  this  deficiency  and  could  also 
be  used  to  rank-order  suppliers  based  on  performance,  to  segment  services  spend  by  high- 
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and  low-performing  suppliers,  and  to  assess  the  performance  of  individual  buying 
activities.  As  such,  rather  than  focusing  on  compliance  as  recommended  by  the  GAO 
(e.g.,  more  oversight),  agencies  could  focus  directly  on  improving  the  perfonnance  of 
service  suppliers  and  the  ultimate  quality  of  those  services  acquired.  Finally,  through  the 
implementation  of  a  supplier  performance-evaluation  system  based  on  a  service  quality 
scale  and  the  implementation  of  the  electronic  submittal  and  aggregation  of  ratings, 
agencies  could  regularly  assess  the  quality  of  acquired  services  at  an  enterprise  level  by 
using  a  services  quality  barometer  or  quality  index  (see,  for  example,  Fomell,  1992). 
Such  an  assessment  would  allow  for  the  identification  of  macro-level  performance  trends 
by  service  type  or  by  top  suppliers,  or  even  allow  for  the  evaluation  of  overall  trends 
resulting  from  policy  changes  or  external  factors. 

E.  LIMITATIONS  OF  THE  STUDY 

This  study  was  limited  in  several  areas.  First,  research  was  perfonned  using  a 
representative  sample  from  a  population.  Because  this  sample  came  solely  from  the  U.S. 
Air  Force,  the  study  findings  may  not  generalize  to  the  DoD,  other  federal  agencies,  or 
the  private  sector.  Additionally,  a  convenience  sample  rather  than  a  random  sample  was 
used.  Convenience  samples  may  introduce  bias  in  respondents.  Second,  service  contract 
administrators  were  asked  to  respond  directly  to  questions  that  were  indicators  of  the  key 
contract  outcomes,  service  quality,  and  regulatory  and  statutory  compliance.  In  some 
circumstances,  the  contract  administrator  may  not  have  had  a  complete  and  accurate 
understanding  of  the  contractor’s  performance  because  this  individual  may  not  have  been 
an  end  user  of  the  service  and  may  not  have  typically  performed  quality  assurance 
functions.  When  answering  questions  regarding  contract  compliance,  contract 
administrators  also  may  not  have  responded  in  a  manner  that  reflected  the  true  level  of 
contract  compliance  if  they  were  not  genuinely  aware  of  all  of  the  regulations  and  statutes 
applicable  to  the  acquisition.  For  example,  an  administrator  who  was  unaware  that 
competition  requirements  applied  to  an  acquisition  would  not  have  considered  the 
acquisition’s  level  of  compliance  with  competition  requirements  when  responding  to  the 
survey.  Additionally,  some  degree  of  socially  desirable  response  bias  may  be  present  for 
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cases  in  which  respondents  were  reluctant  to  tell  the  truth  regarding  sensitive  or 
vulnerable  areas  (e.g.,  contract  compliance).  Finally,  the  data  in  this  study  represent  the 
self-reported  perceptions  of  contracting  professionals.  As  with  all  self-reported  data, 
there  is  a  potential  for  common  method  bias  (Podsakoff  &  Organ,  1986). 

F.  AREAS  FOR  FURTHER  RESEARCH 

This  study  primarily  focused  on  the  contract  outcomes  of  service  quality  and 
compliance  with  regulations  and  statutes.  However,  the  desired  outcomes  of  public 
policy,  such  as  those  implemented  in  the  FAR,  typically  differ  from  those  of  corporate 
policies  in  that  the  goals  of  public  policies  are  numerous  and  occasionally  conflict.  For 
example,  corporate  policies  for  purchasing  within  a  private  organization  may  only  focus 
on  achieving  an  optimal  mix  of  price  (or  total  ownership  costs)  and  quality.  The 
outcomes  of  public  procurement  policy,  especially  policy  at  the  federal  level,  also  include 
maximizing  competition,  furthering  socioeconomic  goals,  and  ensuring  transparency  and 
fairness  in  the  procurement  process.  The  determinants  of  each  of  these  outcomes  and 
their  interactions  could  be  further  explored. 

Furthennore,  this  study  only  examined  the  determinants  of  services  sourcing 
perfonnance  within  the  U.S.  Air  Force.  This  population  could  be  extended  in  future 
research  through  the  application  of  the  model  to  other  federal  populations,  such  as 
another  DoD  Service,  the  DoD  as  a  whole,  or  other  federal  agencies.  If  properly 
executed,  the  results  of  these  studies  could  be  compared  in  an  SEM  package  using  multi¬ 
group  analysis  to  evaluate  the  relative  effectiveness  of  organizational  policies. 
Alternatively,  the  survey  could  be  redeployed,  with  some  modification,  to  procurement 
officers  in  the  private  sector.  Such  an  approach  would  allow  for  a  broader  interpretation 
of  the  model  and  would  allow  for  a  comparative  approach  between  public-  and  private- 
sector  sourcing  of  B2B  services. 

Finally,  future  research  could  address  the  need  to  better  measure  variables  that 
were  measured  in  this  study  using  single-item  scales — the  extent  that  the  contractor 
defined  requirements  and  the  extent  that  PBSA  procedures  were  used.  Future  studies 
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may  either  attempt  to  use  a  multiple-item  scale  to  account  for  measurement  error  or  to 
assess  these  extents  using  objective  measures. 


G.  SUMMARY 

In  conclusion,  this  study  examined  the  determinants  of  services  sourcing 
perfonnance  through  a  quantitative  analysis  of  survey  responses  from  240  U.S.  Air  Force 
contract  administrators,  bridged  a  literature  gap  in  the  measurement  of  customer-defined 
B2B  services,  and  offered  service  contract  practitioners  within  the  U.S.  Air  Force  a 
comprehensive  model  to  better  improve  the  outcomes  of  service  acquisitions.  In  brief, 
service  outcomes  were  found  to  be  most  highly  dependent  on  government-contractor 
communication  and  on  sufficiently  defined  requirements.  Additional  analysis  revealed 
effects  on  outcomes  from  buyer  experience  and  the  turnover  of  acquisition  personnel. 
Significant  second-order  effects  were  also  found  leading  from  the  level  of  commitment  to 
the  acquisition  by  internal  customers  and  to  the  sufficiency  of  procurement  lead-time. 
Finally,  a  non-positive  relationship  was  found  between  the  level  of  compliance  with 
regulations  and  statutes  and  service  quality. 
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APPENDIX  A.  SURVEY  INSTRUMENT 


Responses  to  this  questionnaire  will  be  used  to  analyze  factors  affecting  service  contract 
outcomes.  Your  response  is  requested  no  later  than  15  June  2010. 

This  DoD-funded  research  is  being  conducted  through  the  Naval  Postgraduate  School's 
Graduate  School  of  Business  and  Public  Policy.  Participation  from  professionals,  such  as  you, 
is  very  important  for  the  success  of  this  research.  First,  you  will  be  helping  a  fellow  USAF 
Contracting  Officer  complete  a  rigorous  MBA  program.  More  importantly,  your  response  will 
help  the  researchers  analyze  service  contract  outcomes.  The  results  of  this  study  will 
generate  recommendations  for  further  improvements  to  how  the  DoD  can  better  manage 
services  acquisitions. 

Procedures.  Your  extent  of  participation  in  this  research  involves  only  the  completion  of  this 
questionnaire. 

Synopsis.  This  is  both  an  anonymous  and  voluntary  questionnaire.  (Please  note,  in  order  to 
obtain  consistent  and  usable  results,  it  is  important  that  you  answer  all  questions).  It  will 
take  most  respondents  approximately  15  to  25  minutes  to  complete  the  questionnaire. 

Risks  and  Benefits.  Your  participation  in  this  research  poses  no  known  risk.  There  will  be  no 
personal  benefits  beyond  having  contributed  your  expertise  to  this  important  research.  If 
desired,  you  may  contact  the  researcher  below  if  you  would  like  to  receive  a  report  of  the 
results  of  the  study. 

Confidentiality  &  Privacy  Act.  All  records  of  this  study  will  be  kept  confidential  and,  since 
responses  are  anonymous,  your  privacy  will  not  be  at  risk.  No  information  will  be  publicly 
accessible  which  could  identify  you  as  a  participant.  Responses  will  be  maintained  by  NPS 
for  five  years,  after  which  they  will  be  destroyed. 

Points  of  Contact.  Should  you  have  any  questions  or  comments  regarding  this  survey, 
please  contact  the  Principal  Investigators:  Captain  William  Muir,  USAF,  (831)  236-1179, 
wamuir@nps.edu.  Lieutenant  Colonel  Tim  Hawkins,  USAF,  (831)  656-7647,  DSN  756-7647, 
tqhawkin@nps.edu.  or  Dr.  Gregory  Hildebrandt,  qqhildeb@nps.edu.  Any  other  questions  or 
concerns  may  be  addressed  to  the  IRB  Chair,  Dr.  Angela  O'Dea,  (831)  656-3966,  DSN  756- 
3966,  alodea@nps.edu.  This  survey  was  reviewed  and  approved  by  the  Air  Force  Survey 
Office  (survey  control  number  10-063,  expires  15  June  2010). 

Thank  you  for  your  time  and  your  participation  in  this  effort.  Individuals  such  as  you  help 
researchers  to  advance  both  the  theory  and  practice  of  contract  management. 
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Exploring  USAF  Services  Contracting 
Instructions  and  Consent 


A)  Please  answer  the  questions  honestly  in  your  best  judgment  to  the  best  of  your 
knowledge.  Answers  are  nonattributional;  no  personal  information  will  be  asked  or  recorded 
in  the  resulting  research  report.  In  your  responses,  please  do  not  include  personal 
identifying  information  or  information  restricted  under  operational  security. 

B)  Throughout  the  survey,  you  will  be  required  to  select  one  answer  per  row  only.  Where 
there  are  exceptions  to  this  rule  and  multiple  answers  are  acceptable,  the  questions' 
instructions  will  include  a  specific  instruction  to  "select  all  that  apply". 

C)  You  can  jump  back  and  forth  through  the  questions  if  you  need  to  change  an  answer  or 
want  to  review  your  answers. 

D)  Your  answers  will  be  saved  as  soon  as  you  select  them,  but  they  will  not  be  locked  until 
you  reach  the  last  page  and  click  the  "done"  button. 


*  1.  By  clicking  on  the  "Proceed"  button,  I  am  acknowledging  that  I  have  read  and 
understand  this  information,  that  I  understand  the  nature  and  purpose  of  this 
study  -  including  its  risks  and  benefits,  and  that  I  agree  to  voluntarily 
participate  in  this  online  survey.  I  also  understand  that  I  may  discontinue  at  any 
time  simply  by  exiting  this  website. 

□  Proceed  □  Exit 


Page  |  2 
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Exploring  USAF  Services  Contracting 
Section  1 


INSTRUCTIONS: 

For  the  remainder  of  this  survey,  please  choose  a  single,  specific  service  contract  that  you 
currently  administer  or  have  administered  in  the  past  12  months  and  answer  all  of  the 
questions  with  respect  to  that  contract.  Note:  you  may  choose  a  contract,  blanket  purchase 
agreement,  purchase  order,  or  delivery/task  order  (or  any  other  contractual  instrument 
excluding  the  government  purchase  card  or  imprest  funds)  that  has  been  awarded  for  the 
acquisition  of  services.  Preferably,  choose  a  multi-year  contract,  but  if  your  experience  is 
limited  to  another  type  of  instrument  (per  above),  that  is  sufficient. 

*  1.  What  type  of  service  is  the  contractor  performing? 

□  Research  and  Development  □  Architect-Engineering  □  Maintenance  /  Repair 

□  Modification  of  Equipment  □  Medical  Services  □  Utilities  and  housekeeping 

□  Quality  Control,  Testing,  and  Inspection  □  Education  and  Training 

□  Professional,  Administrative  &  Management  Support  Services  □  Other: 


*  2.  What  is  the  contract  type? 

□  Fixed  Price  □  Cost  Reimbursement  □  Time  and  Materials  □  Labor  Flour 

□  Hybrid  □  Other: 


*  3.  What  letter  is  in  the  ninth  position  of  the  contract  number?  This  letter  indicates 
the  type  of  contract  instrument  used.  For  example,  if  the  contract  number  is  N00023-90- 
D-0009  then  you  would  enter  the  letter  D. 


*  4.  Do  the  services  being  performed  meet  the  definition  of  a  commercial  item,  as 
defined  in  FAR  Part  2?  Select  "Yes"  only  if  your  contract  included  the  clause  FAR 
52.212-4. 

□  Yes  □  No 
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*  5.  What  type  of  incentive(s)/disincentives(s)  does  this  contract  contain  -  if 
any?  (Check  all  that  apply) 

□  Award  Fee  □  Incentive  Fee  □  Award  Term  □  Performance-Based  Payments 

□  Performance  Incentive  □  Delivery  Incentive  □  Liquidated  Damages  clause 

□  This  Contract  Does  Not  Contain  any  Incentives  /  Disincentives  □  Other: 


*  6.  What  is  the  total  dollar  value  of  financial  incentives  available  in  this  contract? 
Enter  0  if  you  did  not  identify  any  incentives  in  the  previous  question.  (Examples: 
amount  of  award  fee  pool,  maximum  incentive  fee  available,  potential  revenue  if  an 
award  term  is  awarded) 


*  7.  Please  fill  in  answers  to  the  following  questions. 

a.  How  many  acquisition  personnel  (contracting  officers  +  contract  specialists  + 
contracting  officer  representatives  +  quality  assurance  evaluators/personnel  + 
inspectors  +  program/project  manager  or  other  active  representative  of  the  user) 
are  currently  assigned  to  this  contract?  Include  only  those  individuals  who  actively, 
routinely  help  manage  the  contract. 


b.  Over  the  life  of  this  contract,  how  many  different  times  have  personnel  turned  over? 
Consider  all  Contract  Administrators,  Contract  Specialists,  Contracting  Officers, 
Contracting  Officer's  Representatives,  and  Quality  Assurance  Personnel  who  actively, 
routinely  helped  manage  the  contract.  (For  example,  if  the  Contract  Administrator 
changed  twice,  the  Contracting  Officer  changed  once,  and  Quality  Assurance 
Personnel  changed  three  times  hen  the  answer  would  be  six) 


c.  How  long  has  this  contract  been  in  effect?  Please  answer  in  months. 


d.  What  is  the  total  dollar  value  of  this  contract,  including  all  option  periods? 
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*  8.  Please  fill  in  the  number  of  days  it  took  to  award  this  contract  -  (i.e.,  from 
the  first  day  that  your  internal  customer  requested  the  service  contract  to  the 
day  that  you  awarded  the  contract) 


*  9.  When  only  considering  the  period  of  time  during  which  you  were  assigned  to 
this  contract,  on  average,  how  many  contract  actions  were  you  working  per  day? 

(A  contract  action  is  defined  herein  as  a  justification  &  approval,  determination  &  finding, 
market  research  report,  solicitation,  award,  modification,  meeting  with  the  contractor, 
meeting  with  internal/Government  stakeholder(s),  site  visit,  negotiation,  ratification, 
quality  deficiency  report,  award  fee  determination,  consultation  with  Legal,  payment, 
contractor  performance  assessment  report,  claim,  bid  protest,  termination,  or  closeout  of 
a  contractual  instrument) 


*  10.  On  a  scale  of  1  to  7,  where  1  represents  "None"  and  7  represents 
"substantial,"  to  what  extent,  if  any,  did  the  contractor  help  define  the 
requirements  prior  to  contract  award?  The  contractor's  help  defining  requirements 
includes  questions  before  and  after  the  issuance  of  the  solicitation,  participation  in 
industry  days,  responses  to  requests  for  information,  responses  to  draft  RFQ/RFP,  and 
any  other  dialogue  with  the  contracting  officer  or  specialist. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 


Page  |  5 


NAVAL  POSTGRADUATE  SCHOOL 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 


153 


Exploring  USAF  Services  Contracting 
Section  4 


*  11.  On  a  scale  of  1  to  7,  where  1  represents  "Much  Worse  than  I  Expected"  and 
7  represents  "Much  Better  than  I  Expected",  rate  the  following: 

a.  The  quality  of  the  contractor's  work. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

b.  When  the  contractor's  management  promises  to  do  something  by  a  certain  time,  it 
does  so. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

c.  How  polite  the  contractor's  employees  are. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

d.  How  much  you  feel  safe  in  your  interactions  with  the  contractor's  employees. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

e.  How  much  you  can  trust  employees  of  the  contractor. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

f.  The  contractor  satisfied  our  need. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

g.  The  contractor's  dependability. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

h.  The  contractor  performed  the  work  we  needed  it  to  do. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

i.  The  contractor's  keeping  of  its  records  accurately. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

j.  The  timeliness  of  the  contractor's  work. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

k.  The  contractor  met  the  requirements  of  the  contract. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

l.  When  you  have  problems,  the  contractor's  management  is  sympathetic  and 
reassuring. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

m.  The  contractor  provides  its  services  at  the  time  it  promises  to  do  so. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

n.  The  adequacy  of  the  support  the  contractor's  employees  receive  from  their  company 
to  do  their  jobs  well. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 
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*  12.  On  a  scale  of  1  to  7,  where  1  represents  "Strongly  Disagree"  and  7 
represents  "Strongly  Agree",  rate  the  following: 

a.  In  our  relationship,  the  government  and  contractor  effectively  communicate 
expectations  for  each  other's  performance. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

b.  I  did  not  have  enough  time  to  award  a  quality  contract. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

c.  The  milestones  for  awarding  this  contract  were  too  aggressive. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

d.  My  leadership  or  my  customer  wanted  this  contract  awarded  too  fast. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

e.  This  contract  is  compliant  with  all  applicable  policy  letters,  the  Federal  Acquisition 
Regulation  and  its  Supplements,  and  procurement  law. 

□  1  d  2  □  3  □  4  □  5  □  6  □  7 

f.  In  our  relationship,  the  government  and  the  contractor  keep  each  other  informed 
of  new  developments. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

g.  There  is  nothing  wrong  with  this  contract. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

h.  This  contract  does  not  violate  applicable  requirements  of  policy  letters,  the  Federal 
Acquisition  Regulation  (FAR),  FAR  Supplements,  or  procurement  law. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

i.  This  contractor  listens  carefully  to  my  requests. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

j.  I  was  not  rushed  to  award  this  contract. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 
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*  13.  On  a  scale  of  1  to  7,  where  1  represents  "Strongly  Disagree"  and  7 
represents  "Strongly  Agree",  rate  the  following: 

a.  The  requirement  was  very  well  defined  in  the  contract. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

b.  The  contract  (including  the  statement  of  work,  performance  work  statement, 
specification,  drawings,  etc.)  defined  the  requirement  very  well. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

c.  If  this  contract  were  to  be  inspected  by  an  independent  organization,  it  would  be 
deemed  compliant. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

d.  In  our  relationship,  the  government  and  contractor  provide  each  other  with 
information  that  helps  both  parties. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

e.  There  were  no  flaws  or  omissions  in  the  definition  of  the  requirement  (including  the 
statement  of  work,  performance  work  statement,  specification,  drawings,  etc.). 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

f.  There  are  aspects  of  this  contract  that,  if  changed,  would  make  it  more  compliant 
with  policy,  laws,  or  regulations. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

g.  I  am  able  to  communicate  my  needs  effectively  to  this  contractor. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

h.  The  requirement,  as  defined  in  the  contract,  expressed  to  the  contractor  exactly 
what  we  needed. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

i.  There  were  no  ambiguities  in  the  definition  of  the  requirement  (including  the 
statement  of  work,  performance  work  statement,  specification,  drawings,  etc.). 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

j.  I  had  sufficient  time  to  get  this  contract  awarded. 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 
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*  14.  On  a  scale  of  1  to  7,  where  1  represents  "Not  at  all"  and  7  represents  "Great 
extent",  rate  the  extent  to  which  this  service  contract  is  performance  based.  (In 

Performance  Based  Services  Acquisition,  requirements  are  defined  in  terms  of  results  to 
be  achieved  rather  than  the  manner  of  performing  the  work.) 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

*  15.  On  a  scale  of  1  to  7,  where  1  represents  "No  Monitoring  of  Supplier"  and  7 
represents  "Extensive  Monitoring  of  Supplier",  rate  the  amount  of  government 
surveillance  of  the  contractor's  performance  in  the  following  areas: 

a.  Service  Quality 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

b.  Timeliness  of  Performance 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

c.  Fulfillment  of  Performance  Requirements  in  the  Statement  of  Work/Performance 
Work  Statement 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

d.  Compliance  With  Contract  Terms  &  Conditions 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

*  16.  The  government  uses  the  following  method(s)  to  perform  surveillance  of  the 
contractor's  performance:  (select  all  that  apply) 

□  Customer  Complaint  □  Periodic  Inspection  □  100-Percent  Inspection 


□  Other: 


*  17.  If  the  means  of  surveillance  involves  inspections  other  than  customer 
complaints,  what  is  the  total  number  of  inspections  that  occur  on  this  contract 
per  year?  (If  surveillance  is  only  accomplished  by  customer  complaint  then  enter  "0" 
into  the  box  provided) 
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*  18.  Considering  all  of  the  quality  assurance  personnel  who  are  assigned  to  this 
contract,  how  many  total  years  of  work  experience  does  the  government's 
quality  assurance  team  have  in  the  technical  area  of  performance  applicable  to 
this  contract?  (For  example,  if  there  are  three  quality  assurance  personnel  assigned  to 
monitor  a  contract  for  aircraft  maintenance  services,  and  their  years  of  experience  of 
inspecting  and  conducting  aircraft  maintenance  are  five,  four,  and  nine  years,  then  the 
total  years  of  experience  for  the  team  would  be  18) 


*  19.  Do  third-party,  contracted  personnel  perform  any  function  of  the 
government's  contract  management  or  quality  assurance  on  this  contract?  (For 
example,  if  the  government  contracted  with  Contractor  A  to  perform  quality  assurance  on 
a  service  contract  performed  by  Contractor  B  then  the  answer  would  be  "Yes") 

□  Yes  □  No 

*  20.  Please  indicate  the  total  number  of  formal  communications  to  the  contractor 
concerning  issue(s)  with  satisfying  contract  requirements.  Communications 
include:  quality  deficiency  reports,  corrective  action  requests,  requests  for 
reperformance,  letters  of  concern,  and  show  cause  notices. 


*  21.  On  a  scale  of  1  to  7,  where  1  represents  "Not  at  All"  and  7  represents  "Great 
Extent",  rate  the  following: 

a.  To  what  extent  is  your  primary  internal  customer  proud  to  be  a  member  of  this 
service  acquisition  team? 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

b.  To  what  extent  does  your  primary  internal  customer  feel  a  sense  of  purpose  when 
the  team  achieves  specific  acquisition  objectives? 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

c.  To  what  extent  does  your  primary  internal  customer  care  about  the  long-term 
success  of  this  acquisition? 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 

d.  To  what  extent  is  your  primary  internal  customer  dedicated  to  ensuring  the 
acquisition  meets  requirements? 

□  1  □  2  □  3  □  4  □  5  □  6  □  7 
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*  22.  By  which  agency  are  you  employed? 

□  United  States  Air  Force  □  United  States  Army  □  United  States  Navy 

□  Other: 

*  23.  At  the  time  you  started  working  on  this  service  contract,  what  was  the 
highest  level  of  Acquisition  Professional  Development  Program  (APDP) 
certification  that  you  held  in  the  area  of  Contracting? 

□  APDP  Level  1  □  APDP  Level  2  □  APDP  Level  3  □  No  Contracting  APDP  Certification 

*  24.  What  is  the  highest  level  of  education  that  you  have  attained? 

□  High  School  Diploma  /  GED  □  Associates  □  Bachelors 

□  Masters  □  Doctoral  /  Professional 

*  25.  Besides  Acquisition  Profession  Development  Program  certifications,  do  you 
hold  any  other  professional  certifications  (such  as  those  granted  by  the  National 
Contract  Management  Association,  the  Institute  for  Supply  Management,  or 
other  professional  associations)? 

□  Yes  □  No 

*  26.  How  many  years  of  experience  do  you  have  in  contracting? 


*  27.  What  is  your  age? 


*  28.  What  is  your  gender? 

□  Male  □  Female 

*  29.  We  appreciate  any  comments  or  feedback  you  can  provide  on  the  topic  of 
services  acquisition. 
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APPENDIX  B.  INVITATION  TO  PARTICIPANTS 


Good  Afternoon  Squadron  Commanders, 

To  support  services  acquisition  research  sponsored  by  Mr.  Roger  Correll  (SAF/AQC)  and  Brig  Gen 
Masiello  (AFPEO/CM) — see  attached  letter,  we  respectfully  request  your  assistance  to  complete 
the  web-based  survey  located  at  the  hyperlink  below.  The  survey  uses  quantitative  methods  to 
study  the  determinants  of  sourcing  performance  for  services  acquisition.  Unit  participation  is 
voluntary;  however,  responses  are  vital  to  research  on  the  management  of  service  acquisitions 
within  the  Air  Force.  Additionally,  this  research  is  part  of  a  student  thesis  project  at  the  Naval 
Postgraduate  School.  For  your  convenience,  a  hard  copy  of  the  survey  is  attached. 

Please  take  the  following  steps: 

1.  Forward  Survey  Link:  Exploring  DoD  Services  Contracting  to  military  and  civilian  personnel  in 
your  unit  who  currently  administer  services  contracts  or  have  administered  service  contracts  in  the 
recent  past.  The  desired  level  of  response  is  ten  (10)  personnel  per  unit. 

2.  Once  forwarded,  please  reply  via  e-mail  to  william.muir@us.af.mil  with  a  count  of  the  number 
of  personnel  you  forwarded  the  survey  to.  This  information  is  critical  to  determine  the  total 
response  rate  for  the  study.  Without  it,  the  statistical  significant  of  the  research  could  be  in 
question. 

Please  ask  personnel  to  complete  the  survey  no  later  than  15  June  2010.  The  survey  should 
take  between  15  and  25  minutes  to  accomplish.  If  you  have  any  questions,  please  contact  Capt 
William  Muir  by  email  to  william.muir@us.af.mil  or  at  (831)  236-1179.  The  time  and  effort  of  you 
and  your  unit  personnel  are  greatly  appreciated. 


Very  Respectfully, 


WILLIAM  A.  MUIR,  Capt,  USAF 
Student,  Naval  Postgraduate  School 
(0)831-236-1179 
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APPENDIX  C.  SAF/AQC  AND  AF  PEO/CM  FETTER  OF 

SUPPORT 


OFFICE  OF  THE  ASSISTANT  SECRETARY 


DEPARTMENT  OF  THE  AIR  FORCE 

WASHINGTON  DC 


MEMORANDUM  FOR  ALMAJCOM-FOA-DRU  (CONTRACTING) 


APR  23  20in 


SUBJECT:  Survey  of  Air  Force  Services  Contract  Administrators 

In  supporting  services  acquisition  research,  my  office  is  sponsoring  a  Naval  Postgraduate 
School  project  to  study  the  determinants  of  services  sourcing  performance.  The  survey  is 
internet-based,  completed  online,  and  does  not  involve  any  physical  copies  or  mailings.  This 
survey  should  take  less  than  thirty  (30)  minutes  to  complete  and  will  provide  insight  into  the 
relationships  between  service  performance,  contracting  compliance,  and  the  following  topics: 

•  Financial  Incentives  •  Requirements  Definition 

•  Surveillance  Methods  •  Education  &  Training 

•  CO  /  CA  Manpower  •  Personnel  Workload 

•  Relational  Exchange 

Note:  The  survey  is  designed  to  be  completed  by  individuals  who  currently  administer  service 
contracts  or  have  done  so  in  the  recent  past.  Approximately  ten  (10)  responses  are  requested 
from  each  contracting  unit. 

The  survey  was  approved  by  the  Air  Force  Survey  Office  (control  number  10-063).  Our 
office  fully  supports  this  research  and  requests  your  cooperation  to  complete  the  surveys  by  1 5  Jun 
2010. 


The  point  of  contract  for  the  survey  is  Captain  William  Muir,  who  can  be  reached  at 
(831)  236-1 179  or  via  e-mail  to  william.muir@us.af.mil. 


Deputy  Assistant  Secretary  (Contracting) 
Assistant  Secretary  (Acquisition) 


WENDY  M.WIASIELLO,  Brig  Gen,  USAF 
Air  Force  Program  Executive  Officer 
for  Combat  and  Mission  Support 
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APPENDIX  D.  DESCRIPTIVE  STATISTICS  FOR  TESTS  OF  BIAS 


Variable 

Group 

N 

Mean 

St.  dev. 

S.  E. 

95%  Cl  for  mean 
Lower  Upper 
bound  bound 

Min. 

Max. 

Age 

1 

80.00 

40.81 

11.92 

1.33 

38.16 

43.46 

22.00 

65.00 

2 

80.00 

42.01 

13.04 

1.46 

39.11 

44.91 

20.00 

65.00 

3 

80.00 

43.36 

11.34 

1.27 

40.83 

45.88 

24.00 

65.00 

Total 

240.00 

42.06 

12.11 

0.78 

40.52 

43.60 

20.00 

65.00 

Gender 

1 

80.00 

0.55 

0.50 

0.06 

0.44 

0.66 

0.00 

1.00 

2 

80.00 

0.53 

0.50 

0.06 

0.41 

0.64 

0.00 

1.00 

3 

80.00 

0.48 

0.50 

0.06 

0.36 

0.59 

0.00 

1.00 

Total 

240.00 

0.52 

0.50 

0.03 

0.45 

0.58 

0.00 

1.00 

Experience 

1 

80.00 

11.80 

9.59 

1.07 

9.67 

13.93 

0.83 

38.00 

2 

80.00 

10.74 

9.90 

Ell 

8.53 

12.94 

0.00 

35.00 

3 

80.00 

12.88 

10.32 

1.15 

10.58 

15.18 

0.42 

41.00 

Total 

240.00 

11.81 

9.94 

0.64 

10.54 

13.07 

0.00 

41.00 

Education 

1 

80.00 

2.00 

0.90 

0.10 

1.80 

2.20 

0.00 

3.00 

2 

80.00 

1.95 

1.07 

0.12 

1.71 

2.19 

0.00 

4.00 

3 

80.00 

1.98 

0.97 

0.11 

1.76 

2.19 

0.00 

4.00 

Total 

240.00 

1.98 

0.98 

0.06 

1.85 

2.10 

0.00 

4.00 

Sufficient 

1 

80.00 

26.01 

6.03 

0.67 

24.67 

27.35 

12.00 

35.00 

requirement 

2 

80.00 

25.79 

6.00 

0.67 

24.45 

27.12 

11.00 

35.00 

definition 

3 

80.00 

25.41 

6.70 

0.75 

23.92 

26.90 

8.00 

35.00 

Total 

240.00 

25.74 

6.23 

0.40 

24.95 

26.53 

8.00 

35.00 

Regulatory 

1 

80.00 

27.58 

5.69 

0.64 

26.31 

28.84 

7.00 

35.00 

and 

2 

80.00 

28.31 

4.81 

0.54 

27.24 

29.38 

14.00 

35.00 

statutory 

3 

80.00 

28.19 

4.88 

0.55 

27.10 

29.27 

8.00 

35.00 

compliance 

Total 

240.00 

28.03 

5.13 

0.33 

27.37 

28.68 

7.00 

35.00 

Service 

1 

80.00 

80.28 

13.64 

1.53 

77.24 

83.31 

48.00 

98.00 

quality 

2 

80.00 

77.60 

15.08 

1.69 

74.24 

80.96 

39.00 

98.00 

3 

80.00 

79.86 

14.97 

1.67 

76.53 

83.19 

42.00 

98.00 

Total 

240.00 

79.25 

14.57 

0.94 

77.39 

81.10 

39.00 

98.00 
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APPENDIX  E.  SCREE  PLOT  OF  EXOGENOUS  FACTORS 


10.000 
9.000 
8.000 
7.000 
6.000 
5.000 
4.000 
3.000 
2.000 
1.000 
.000 

1  2  3  4  5  6  7  8  9  10  11  12 


✓ 

✓ 


✓ 

✓ 


100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 


Component 


Eingenvalue 


Cumulative  percent  of  explained  variance 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


168 


Eingenvalue 


APPENDIX  F.  SCREE  PLOT  OF  ENDOGENOUS  FACTORS 


Component 


Eingenvalue 


Cumulative  percent  of  explained  variance 
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x2 

P  value 

Z2/df 

CFI 

TLI 

RMSEA 

SRMR 

Measurement  model 

383.71 

<0.01 

1.38 

0.98 

0.98 

0.04 

0.04 

Fitness  criterion 

>0.05 

<3.00 

>0.95 

>0.95 

<0.08 

<0.08 
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APPENDIX  I.  RESIDUAL  VARIANCE  TABLES 


Exogenous  Measurement  Model 


Construct 

Item 

Residual 

variance 

Standard 

error 

Critical 

ratio 

P  value 

Communication 

REa 

0.34 

0.06 

5.29 

<0.01 

REb 

0.41 

0.06 

6.40 

<0.01 

REd 

0.69 

0.06 

11.65 

<0.01 

Amount  of 

ATla 

0.24 

0.03 

7.56 

<0.01 

surveUance 

ATlc 

0.12 

0.02 

5.10 

<0.01 

ATld 

0.13 

0.02 

5.35 

<0.01 

Procurement  lead- 

RBVlc 

0.63 

0.06 

10.95 

<0.01 

time  sufficiency 

RBVld 

0.26 

0.06 

4.57 

<0.01 

RBVle 

0.30 

0.06 

5.28 

<0.01 

Internal  customer 

ICCa 

0.17 

0.03 

5.72 

<0.01 

commitment 

ICCb 

0.10 

0.03 

3.52 

<0.01 

ICCd 

0.44 

0.05 

9.44 

<0.01 

Endogenous  Measurement  Model 

Construct 

Item 

Residual 

variance 

Standard 

error 

Critical 

ratio 

P  value 

Service  quality 

SQa 

0.34 

0.04 

9.07 

<0.01 

SQc 

0.13 

0.02 

7.06 

<0.01 

SQd 

0.17 

0.02 

7.55 

<0.01 

SQf 

0.47 

0.05 

10.31 

<0.01 

SQi 

0.34 

0.04 

9.09 

<0.01 

SQj 

0.17 

0.02 

7.66 

<0.01 

SQk 

0.16 

0.02 

7.40 

<0.01 

SQm 

0.18 

0.02 

7.62 

<0.01 

Regulatory  and 

RSCa 

0.10 

0.07 

1.32 

0.19 

statutory  compliance 

RSCc 

0.32 

0.06 

5.02 

<0.01 

RSCe 

0.85 

0.05 

18.13 

<0.01 

Requirement 

RDla 

0.24 

0.04 

5.96 

<0.01 

definition  sufficiency 

RDlb 

0.16 

0.04 

4.16 

<0.01 

RDle 

0.48 

0.05 

9.47 

<0.01 
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APPENDIX  K.  CORRELATION  MATRIX 


Variable 

Mean 

S.D. 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(ID 

(12) 

(13) 

(14) 

(15) 

(1) 

REa 

5.73 

1.18 

1.00 

(2) 

REb 

5.65 

1.29 

.623" 

1.00 

(3) 

REd 

5.88 

1.17 

.452" 

4=4= 

.451 

1.00 

(4) 

ATla 

5.29 

1.49 

.181" 

.126 

.123 

1.00 

(5) 

ATlc 

5.34 

1.53 

.228" 

4=4= 

.172 

4=  * 

.170 

4:4: 

.818 

1.00 

(6) 

ATld 

5.44 

1.48 

.199" 

4= 

.131 

4:4c 

.203 

4=4= 

.819 

4=  4= 

.875 

1.00 

(7) 

RBV 

4.45 

1.93 

.290" 

.206 

4= 

.150 

.015 

.024 

.056 

1.00 

(8) 

RBV 

4.13 

1.84 

.306" 

.258 

4=4= 

.224 

.036 

.084 

.042 

4=4= 

.517 

1.00 

(9) 

RBV 

4.45 

1.69 

.291" 

4=4= 

.208 

4:4: 

.186 

.083 

.126 

.107 

4=4= 

.520 

4=4= 

.725 

1.00 

(10) 

ICCa 

5.25 

1.47 

.418" 

4=4= 

.406 

4:4: 

.248 

4=4= 

.238 

4:4: 

.277 

4:4: 

.191 

4=4= 

.222 

4=4= 

.309 

4:4: 

.255 

1.00 

(11) 

ICCb 

5.40 

1.37 

.448" 

.442 

4=4= 

.250 

4:4: 

.189 

4:4: 

.245 

4=4= 

.173 

4=4= 

.231 

4=4= 

.329 

.315** 

4:4: 

.865 

1.00 

(12) 

ICCd 

5.67 

1.33 

.384" 

.399 

.207" 

4:4: 

.208 

4=  4= 

.228 

.177" 

4:4: 

.290 

4c* 

.337 

4=4= 

.306 

4=4= 

.678 

4=  4: 

.711 

1.00 

(13) 

SQa 

5.38 

1.34 

.548" 

4=4= 

.531 

4:4: 

.311 

.099 

4s 

.153 

.093 

.104 

4=4= 

.249 

4:4: 

.184 

4=4= 

.434 

4=4= 

.401 

4=4= 

.324 

1.00 

(14) 

SQc 

5.77 

1.17 

.573" 

4=4= 

.539 

4=4: 

.357 

.172" 

4:4: 

.226 

4=4= 

.206 

.063 

4=4= 

.192 

.107 

4=4= 

.372 

*4= 

.335 

4=4= 

.252 

4=4= 

.734 

1.00 

(15) 

SQd 

5.70 

1.18 

.575" 

4=4= 

.592 

.377" 

.  151* 

4=  4= 

.219 

4:4: 

.179 

.043 

.149* 

.107 

4=4= 

.332 

4=4= 

.292 

4=4= 

.247 

4=4= 

.743 

4=4= 

.841 

1.00 

(16) 

SQf 

5.62 

1.17 

.506" 

4=4= 

.537 

*4: 

.276 

.107 

4=4= 

.206 

4= 

.146 

.137* 

4=4= 

.215 

* 

.131 

4=4= 

.349 

4=4= 

.344 

4=4= 

.310 

4=4= 

.577 

4=4= 

.691 

.647** 

(17) 

SQi 

5.49 

1.30 

.545" 

.559 

.354" 

.111 

4=4= 

.189 

* 

.157 

.094 

4=4= 

.185 

.190" 

4=4= 

.371 

4=4= 

.349 

4=4= 

.281 

4=4= 

.705 

4=4= 

.749 

.753** 

(18) 

SQj 

5.78 

1.23 

.549" 

.580 

4=4= 

.320 

.  151* 

4=  4= 

.240 

4:4: 

.212 

.044 

.166* 

.102 

4=4= 

.369 

4=4= 

.356 

.287** 

4=4= 

.734 

4=4= 

.848 

.819" 

(19) 

SQk 

5.76 

1.18 

.576" 

4=4= 

.544 

.317" 

.  144* 

4:4: 

.216 

4:4: 

.195 

.040 

4=4: 

.199 

.119 

4:4: 

.395 

4=4= 

.373 

4=4= 

.294 

4=4= 

.744 

4=4= 

.891 

.824**! 

(20) 

SQm 

5.71 

1.25 

.563" 

4=4= 

.586 

4=4= 

.299 

.  164* 

*4= 

.196 

4=4= 

.169 

.074 

.164* 

.103 

4=4= 

.374 

4=4= 

.336 

4=4= 

.268 

4=4= 

.751 

4=4= 

.831 

.868" 

(21) 

RSCa 

6.19 

1.16 

.467" 

4=4= 

.492 

.357" 

.127* 

4=  4= 

.206 

4:4: 

.212 

.139* 

4=4= 

.190 

4:4: 

.234 

4=4= 

.276 

4=4= 

.340 

4=4= 

.304 

4=4= 

.254 

4=4= 

.297 

.271"! 

(22) 

RSCc 

6.23 

1.18 

.328" 

4=4= 

.407 

4=4= 

.260 

.071 

.119 

4= 

.148 

.127* 

4=4= 

.183 

4:4: 

.176 

4:4: 

.234 

4=4= 

.322 

4=4= 

.256 

4=4= 

.199 

4=4= 

.214 

.221**1 

(23) 

RSCe 

4.80 

1.75 

.171" 

4=4= 

.169 

4=4= 

.184 

.030 

.068 

.047 

.120 

.151* 

.110 

.036 

.090 

.110 

.068 

.031 

.101 

(24) 

RDla 

5.31 

1.49 

.438" 

4=4= 

.377 

.335" 

.045 

4= 

.155 

* 

.143 

4=4= 

.252 

4=4= 

.349 

4=  4: 

.343 

4=4= 

.344 

4=4= 

.366 

4=4= 

.350 

4=4= 

.391 

4=4= 

.346 

.349**| 

(25) 

RDlb 

5.25 

1.45 

.460" 

.383 

4:4: 

.323 

.049 

4= 

.146 

.118 

*4= 

.254 

4=4= 

.320 

4=4* 

.327 

4:4: 

.414 

4=4= 

.390 

4=4= 

.430 

4=4= 

.386 

4=4= 

.341 

.356** 

(26) 

RDle 

4.91 

1.59 

.396" 

.276 

4=4= 

.261 

.069 

.089 

.067 

4=4= 

.208 

4=4= 

.259 

.320** 

4=4= 

.362 

4=4= 

.383 

4=4= 

.395 

4=4= 

.400 

4=4= 

.308 

.289** 

(27) 

AT2 

5.60 

1.63 

.171" 

4=4= 

.191 

4= 

.143 

4=4= 

.183 

4:  * 

.257 

4=4= 

.233 

.091 

.072 

4:4: 

.176 

4:4: 

.167 

4=4= 

.195 

4= 

.144 

.100 

.119 

.115 

(28) 

RD2 

2.57 

1.74 

.033 

.086 

-.012 

.077 

.057 

.032 

-.083 

.019 

-.009 

.094 

.087 

.040 

.015 

.064 

.066  ! 

(29) 

RBV 

1.22 

1.17 

-.017 

-.091 

.031 

-.054 

-.039 

-.044 

.077 

.002 

-.017 

-.003 

-.077 

-.028 

.008 

.076 

.023  | 

(30) 

CBV 

11.66 

9.88 

-.036 

.018 

4= 

.128 

-.022 

-.009 

.028 

-.161* 

4= 

-.137 

4= 

-.178 

-.014 

.007 

-.022 

.085 

.099 

.108  ! 

(31) 

CBV 

1.68 

1.06 

.014 

.013 

.125 

.014 

.063 

.086 

-.146* 

4C5 

-.172 

4= 

-.177 

-.101 

-.079 

-.108 

.032 

.011 

.081  | 

(32) 

CBV 

1.98 

0.98 

-.052 

.034 

-.013 

.041 

.058 

.059 

-.050 

-.122 

-.075 

.015 

.009 

.040 

-.054 

-.044 

-.045 

“Correlation  is  significant  at  the  0.01  level  (2-tailed),  *Correlation  is  significant  at  the  0.05  level  (2-tailed). 


Variable 

(16) 

(17) 

(18) 

09) 

(20) 

(21) 

(22) 

(23) 

(24) 

(25) 

(26) 

(27) 

(28) 

(29) 

(30) 

(31) 

(32) 

(16) 

SQf 

1.00 

(17) 

SQi 

.642** 

1.00 

(18) 

SQj 

.665** 

*  * 

.754 

1.00 

(19) 

SQk 

.638** 

** 

.727 

.834** 

1.00 

(20) 

SQm 

.666** 

** 

.694 

.830** 

** 

.823 

1.00 

(21) 

RSCa 

.236** 

** 

.291 

.308** 

** 

.287 

** 

.253 

1.00 

(22) 

RSCc 

.206** 

*  * 

.230 

.234** 

%  3{e 

.216 

>lc  * 

.247 

.785 

1.00 

(23) 

RSCe 

.034 

.031 

.057 

.036 

.054 

^  $ 

.364 

** 

.357 

1.00 

(24) 

RDla 

.306** 

** 

.360 

.332** 

** 

.378 

** 

.382 

** 

.356 

.271** 

* 

.141 

1.00 

(25) 

RDlb 

.304** 

*  * 

.345 

.369** 

** 

.371 

s(:  * 

.396 

$  $ 

.356 

** 

.281 

.168** 

*  * 

.801 

1.00 

(26) 

RDle 

.228** 

*  * 

.308 

.312** 

** 

.332 

*  * 

.299 

.339** 

** 

.261 

.209** 

*  * 

.620 

** 

.665 

1.00 

(27) 

AT2 

.082 

* 

.128 

.125 

.160* 

.080 

.162* 

.130* 

* 

.141 

.112 

** 

.181 

.105 

1.00 

(28) 

RD2 

.026 

* 

.133 

.086 

.068 

.032 

.020 

.056 

-.060 

-.082 

-.078 

-.019 

* 

-.155 

1.00 

(29) 

RBV 

.060 

.023 

.083 

.034 

.044 

-.114 

-.163* 

-.124 

-.024 

.007 

-.068 

-.061 

-.073 

1.00 

(30) 

CBV 

.068 

.031 

.149* 

.063 

.082 

.013 

.061 

.014 

.005 

-.034 

-.013 

.024 

-.085 

.011 

1.00 

(31) 

CBV 

.022 

.048 

.061 

.032 

.028 

-.038 

.006 

-.008 

-.005 

-.062 

-.027 

.028 

-.027 

.122 

** 

.601 

1.00 

(32) 

CBV 

.011 

-.035 

-.059 

-.077 

-.012 

.015 

.030 

.020 

.016 

.030 

-.058 

-.008 

-.035 

.037 

-.045 

.135* 

1.00 

**Correlation  is  significant  at  the  0.01  level  (2-tailed),  *Correlation  is  significant  at  the  0.05  level  (2-tailed). 
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APPENDIX  L.  MPLUS  INPUT  FOR  HYPOTHESIZED  MODEL 


TITLE:  HYPOTHESIZED  MODEL 

DATA:  FILE  IS  "DATASET.DAT"; 

VARIABLE:  NAMES  ARE  REa  REb  REd  AT  la  ATlc  AT  Id  RBVlc  RBVld  RBVle 

ICCa  ICCb  ICCd  SQa  SQc  SQd  SQf  SQi  SQj  SQk  SQm  RSCa  RSCc 
RSCe  RDla  RDlb  RDle  AT2  RD2  RBV2  CBV1  CBV2  CBV3; 
MODEL:  RE  BY  REa  REb  REd; 

RBV1  BY  RBVlc  RBVld  RBVle; 

ICC  BY  ICCa  ICCb  ICCd; 

ATI  BY  ATlaATlc  ATld; 

SQ  BY  SQa  SQc  SQd  SQf  SQi  SQj  SQk  SQm; 

RSC  BY  RSCa  RSCc  RSCe; 

RD1  BY  RDla  RDlb  RDle; 

SQ  ON  RE  ATI  AT2  RD2  RD1  RBV1  RBV2  RSC; 

RSC  ON  RE  RD1  RBV1  RBV2  CBV1  CBV2  CBV3  ICC; 

RD1  ON  RD2  RBV1  CBV1  CBV2  CBV3  ICC; 

MODEL  INDIRECT: 

SQ  IND  RE; 

SQ  INDRD1; 

SQ  IND  RD2; 

SQ  IND  RBV1; 

SQ  IND  RBV2; 

RSC  IND  RBV1; 

RSC  IND  CBV1; 

RSC  IND  CBV2; 

RSC  IND  CBV3; 

RSC  IND  ICC; 

ANALYSIS:  TYPE  IS  GENERAL; 

ESTIMATOR  IS  ML; 

ITERATIONS  =  1000; 

CONVERGENCE  =  0.00005; 

OUTPUT:  STANDARDIZED; 
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APPENDIX  M.  MPLUS  OUTPUT  FOR  HYPOTHESIZED  MODEL 


Mplus  VERSION  6 
MUTHEN  &  MUTHEN 
10/10/2010  5:05  PM 

HYPOTHESIZED  MODEL 


SUMMARY  OF  ANALYSIS 


Number  of  groups  1 

Number  of  observations  240 


Number  of  dependent  variables  26 
Number  of  independent  variables  6 
Number  of  continuous  latent  variables  7 

Observed  dependent  variables 


Continuous 

REA 

REB 

RED 

ATI  A 

AT  1C 

AT  ID 

RBV1C 

RBV1D 

RBV1E 

ICCA 

ICCB 

ICCD 

SQA 

SQC 

SQD 

SQF 

SQI 

SQ  J 

SQK 

SQM 

RSCA 

RSCC 

RSCE 

RD1A 

RD1B 

RD1E 

Observed  independent 

variables 

AT  2 

RD2 

RBV2 

CBV1 

CBV2 

CBV3 

Continuous  latent  variables 

RE 

RBV1 

ICC 

ATI 

SQ 

RSC 

RD1 


Estimator 
Information  matrix 
Maximum  number  of  iterations 
Convergence  criterion 

Maximum  number  of  steepest  descent  iterations 


ML 

OBSERVED 
1000 
0 . 500D-04 
20 


Input  data  file(s) 
DATASET . DAT 


Input  data  format  FREE 

THE  MODEL  ESTIMATION  TERMINATED  NORMALLY 
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TESTS  OF  MODEL  FIT 


Chi-Square  Test  of  Model  Fit 

Value  532 . 257 

Degrees  of  Freedom  403 

P-Value  0.0000 

Chi-Square  Test  of  Model  Fit  for  the  Baseline  Model 
Value  5395.376 

Degrees  of  Freedom  481 

P-Value  0.0000 


CFI/TLI 

CFI 
TLI 

Loglikelihood 

HO  Value 
HI  Value 

Information  Criteria 

Number  of  Free  Parameters 
Akaike  (AIC) 

Bayesian  (BIC) 

Sample-Size  Adjusted  BIC 
(n*  =  (n  +  2)  /  24) 

RMSEA  (Root  Mean  Square  Error  Of  Approximation) 

Estimate  0.037 

90  Percent  C.I.  0.028  0.045 

Probability  RMSEA  <=  .05  0.998 

SRMR  (Standardized  Root  Mean  Square  Residual) 

Value  0.047 


130 

22702.242 
23154.725 
22742 . 657 


-11221.121 
-10954 . 992 


0 . 974 
0.969 
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MODEL  RESULTS 


Two-Tailed 

P-Value 


Estimate  S.E.  Est./S.E. 


RE  BY 


REA 

1.000 

0.000 

999.000 

999.000 

REB 

1.104 

0.087 

12 . 625 

0.000 

RED 

0 . 673 

0.081 

8.275 

0.000 

RBV1 

BY 

RBV1C 

1.000 

0.000 

999.000 

999.000 

RBV1D 

1.293 

0.135 

9.616 

0.000 

RBV1E 

1 .193 

0.124 

9.621 

0.000 

ICC 

BY 

ICCA 

1.000 

0.000 

999.000 

999.000 

ICCB 

0 . 955 

0.042 

22 . 998 

0.000 

ICCD 

0.747 

0.050 

15.058 

0.000 

ATI 

BY 

ATI  A 

1.000 

0.000 

999.000 

999.000 

AT  1C 

1 . 102 

0.051 

21.588 

0.000 

AT  ID 

1.056 

0.049 

21.551 

0.000 

SQ 

BY 

SQA 

1.000 

0.000 

999.000 

999.000 

SQC 

0 . 998 

0.054 

18.550 

0.000 

SQD 

0 . 985 

0.055 

18.031 

0.000 

SQF 

0.782 

0.061 

12 .889 

0.000 

SQI 

0.963 

0.064 

15.046 

0.000 

SQ  J 

1.025 

0.057 

17 . 871 

0.000 

SQK 

0 . 991 

0.055 

18.147 

0.000 

SQM 

1.035 

0.058 

17 .888 

0.000 

RSC 

BY 

RSCA 

1.000 

0.000 

999.000 

999.000 

RSCC 

0 . 854 

0.066 

12 . 919 

0.000 

RSCE 

0 . 600 

0.106 

5.688 

0.000 

RD1 

BY 

RD1A 

1.000 

0.000 

999.000 

999.000 

RD1B 

1.027 

0.059 

17.410 

0.000 

RD1E 

0 .892 

0.069 

12 . 968 

0.000 

SQ 

ON 

RE 

1.066 

0.129 

8.256 

0.000 

ATI 

0.040 

0.045 

0.885 

0.376 

RD1 

0 . 150 

0.055 

2.728 

0.006 

RBV1 

-0.183 

0.063 

-2 . 928 

0.003 

RSC 

-0.245 

0.074 

-3.326 

0.001 

RSC 

ON 

RE 

0 . 681 

0.114 

5.959 

0.000 

RD1 

0 . 128 

0.066 

1 . 944 

0.052 

RBV1 

-0.025 

0.073 

-0.341 

0.733 

ICC 

-0.032 

0.065 

-0.494 

0 . 621 
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RD1 


ON 


RBV1 

0.343 

0.086 

3.983 

0.000 

ICC 

0.363 

0.067 

5.395 

0.000 

SQ 

ON 

AT  2 

-0.029 

0.035 

-0 . 827 

0.408 

RD2 

0.017 

0.032 

0.527 

0.598 

RBV2 

0.071 

0.048 

1.497 

0.134 

RSC 

ON 

RBV2 

-0.089 

0.055 

-1 . 613 

0.107 

CBV1 

0.004 

0.008 

0.538 

0.591 

CBV2 

-0.070 

0.076 

-0 . 922 

0.356 

CBV3 

0.058 

0.064 

0 . 902 

0.367 

RD1 

ON 

RD2 

-0.082 

0.044 

-1 . 874 

0.061 

CBV1 

0.001 

0.010 

0 . 149 

0 .882 

CBV2 

0.054 

0.091 

0.588 

0.556 

CBV3 

0.042 

0.079 

0.531 

0.596 

RBV1 

WITH 

RE 

0.477 

0.105 

4.525 

0.000 

ICC 

WITH 

RE 

0.725 

0.110 

6.608 

0.000 

RBV1 

0 . 645 

0.136 

4.730 

0.000 

ATI 

WITH 

RE 

0.319 

0.094 

3.403 

0.001 

RBV1 

0.162 

0.113 

1.427 

0.154 

ICC 

0.451 

0.125 

3.598 

0.000 

AT  2 

WITH 

RE 

0.357 

0.107 

3.344 

0.001 

RBV1 

0.290 

0.134 

2.158 

0.031 

ICC 

0.433 

0 . 141 

3.066 

0.002 

ATI 

0.535 

0.136 

3.932 

0.000 

RD2 

WITH 

RE 

0 . 105 

0.113 

0 . 932 

0.351 

RBV1 

-0.020 

0 . 140 

-0.146 

0 .884 

ICC 

0.215 

0.150 

1.431 

0.152 

ATI 

0 . 125 

0 . 144 

0 . 870 

0.384 

RBV2 

WITH 

RE 

-0.060 

0.077 

-0.778 

0.437 

RBV1 

0.007 

0.094 

0.071 

0 . 943 

ICC 

-0.079 

0.101 

-0.778 

0.437 

ATI 

-0.072 

0.097 

-0.746 

0.456 

CBV1 

WITH 

RE 

0.363 

0 . 626 

0.580 

0.562 

RBV1 

-2.342 

0 . 824 

-2 . 842 

0.004 

ICC 

-0.053 

0 . 854 

-0.062 

0 . 951 

ATI 

0.037 

0.819 

0.045 

0 . 964 
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WITH 


CBV2 


RE 

0.022 

0.067 

0.324 

0.746 

RBV1 

-0.262 

0.088 

-2 . 966 

0.003 

ICC 

-0 . 140 

0.092 

-1.528 

0.127 

ATI 

0.092 

0.088 

1.049 

0.294 

CBV3  WITH 

RE 

-0.034 

0.062 

-0.548 

0.584 

RBV1 

-0 . 126 

0.080 

-1.590 

0.112 

ICC 

0.021 

0.085 

0.253 

0 . 800 

ATI 

0.075 

0.081 

0 . 927 

0.354 

Intercepts 

REA 

5.729 

0.074 

77.242 

0.000 

REB 

5.654 

0.081 

69.559 

0.000 

RED 

5.883 

0.074 

79.160 

0.000 

AT1A 

5.292 

0.093 

56.793 

0.000 

AT  1C 

5.342 

0.095 

56.259 

0.000 

AT  ID 

5.442 

0.092 

59.376 

0.000 

RBV1C 

4.454 

0.122 

36.496 

0.000 

RBV1D 

4 . 129 

0.115 

35.957 

0.000 

RBV1E 

4.446 

0.105 

42.267 

0.000 

ICCA 

5.250 

0.092 

57.090 

0.000 

ICCB 

5.400 

0.085 

63.504 

0.000 

ICCD 

5.671 

0.084 

67 . 610 

0.000 

SQA 

5.396 

0.257 

21.033 

0.000 

SQC 

5.791 

0.252 

22 . 937 

0.000 

SQD 

5.724 

0.250 

22 . 939 

0.000 

SQF 

5.633 

0.203 

27.702 

0.000 

SQI 

5.512 

0.247 

22.303 

0.000 

SQJ 

5.800 

0.260 

22.320 

0.000 

SQK 

5.783 

0.251 

23.024 

0.000 

SQM 

5.734 

0.263 

21 . 835 

0.000 

RSCA 

6.255 

0.183 

34 . 116 

0.000 

RSCC 

6.284 

0.162 

38 . 831 

0.000 

RSCE 

4 . 838 

0.151 

32.044 

0.000 

RD1A 

5.333 

0.249 

21.392 

0.000 

RD1B 

5.271 

0.254 

20.727 

0.000 

RD1E 

4 . 926 

0.229 

21.484 

0.000 

Variances 

RE 

0 . 877 

0.125 

7.018 

0.000 

RBV1 

1.432 

0.287 

4 . 992 

0.000 

ICC 

1 . 805 

0.200 

9.041 

0.000 

ATI 

1 . 695 

0.200 

8.478 

0.000 

Residual  Variances 

REA 

0.503 

0.063 

7 . 962 

0.000 

REB 

0.589 

0.076 

7.789 

0.000 

RED 

0 . 956 

0.093 

10.271 

0.000 

ATI  A 

0.528 

0.060 

8.793 

0.000 

AT  1C 

0.275 

0.051 

5.436 

0.000 

AT  ID 

0.281 

0.048 

5.911 

0.000 

RBV1C 

2.266 

0.232 

9.746 

0.000 

RBV1D 

0 . 974 

0.164 

5.958 

0.000 

RBV1E 

0.792 

0.137 

5.766 

0.000 

ICCA 

0.357 

0.056 

6.344 

0.000 
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ICCB 

0.209 

0.047 

4.485 

0.000 

ICCD 

0.756 

0.076 

9.908 

0.000 

SQA 

0 . 610 

0.059 

10.275 

0.000 

SQC 

0 . 183 

0.021 

8 . 605 

0.000 

SQD 

0.229 

0.025 

9.138 

0.000 

SQF 

0 . 642 

0.061 

10.563 

0.000 

SQI 

0.570 

0.056 

10.271 

0.000 

SQJ 

0.263 

0.028 

9.318 

0.000 

SQK 

0.219 

0.024 

9.023 

0.000 

SQM 

0.270 

0.029 

9.240 

0.000 

RSCA 

0.088 

0.076 

1.145 

0.252 

RSCC 

0.472 

0.070 

6.723 

0.000 

RSCE 

2.592 

0.242 

10.709 

0.000 

RD1A 

0.533 

0.080 

6.668 

0.000 

RD1B 

0.338 

0.074 

4.585 

0.000 

RD1E 

1.187 

0.123 

9.617 

0.000 

SQ 

0.398 

0.074 

5.354 

0.000 

RSC 

0.746 

0.104 

7 . 154 

0.000 

RD1 

1 . 110 

0 . 144 

7.715 

0.000 
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STANDARDIZED  MODEL  RESULTS 


STDYX  Standardization 

Two-Tailed 


Estimate 

S.E. 

Est./S.E. 

P-Value 

RE 

BY 

REA 

0.797 

0.031 

25.975 

0.000 

REB 

0 . 803 

0.030 

26.436 

0.000 

RED 

0.542 

0.051 

10 . 689 

0.000 

RBV1 

BY 

RBV1C 

0 . 622 

0.046 

13.601 

0.000 

RBV1D 

0.843 

0.031 

27 . 643 

0.000 

RBV1E 

0 . 849 

0.030 

28.030 

0.000 

ICC 

BY 

ICCA 

0 . 914 

0.016 

57 . 963 

0.000 

ICCB 

0 . 942 

0.014 

66.622 

0.000 

ICCD 

0.756 

0.030 

24 . 973 

0.000 

ATI 

BY 

ATI  A 

0.873 

0.018 

48 . 857 

0.000 

AT  1C 

0 . 939 

0.013 

74 . 629 

0.000 

AT  ID 

0 . 933 

0.013 

72.045 

0.000 

SQ 

BY 

SQA 

0 . 809 

0.023 

34.756 

0.000 

SQC 

0.929 

0.010 

90.246 

0.000 

SQD 

0 . 911 

0.012 

74.245 

0.000 

SQF 

0.724 

0.032 

22 . 933 

0.000 

SQI 

0 . 808 

0.023 

34.526 

0.000 

SQ  J 

0 . 906 

0.013 

71.239 

0.000 

SQK 

0 . 915 

0.012 

77 . 684 

0.000 

SQM 

0 . 906 

0.013 

70 . 654 

0.000 

RSC 

BY 

RSCA 

0 . 966 

0.030 

31 . 801 

0.000 

RSCC 

0 . 808 

0.033 

24.168 

0.000 

RSCE 

0.381 

0.059 

6.491 

0.000 

RD1 

BY 

RD1A 

0.871 

0.023 

38.322 

0.000 

RD1B 

0 . 916 

0.020 

45.241 

0.000 

RD1E 

0.727 

0.034 

21.094 

0.000 
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SQ  ON 


RE 

0 . 930 

0.079 

11.773 

0.000 

ATI 

0.048 

0.055 

0 .887 

0.375 

RD1 

0 . 181 

0.066 

2.716 

0.007 

RBV1 

-0.204 

0.068 

-2 . 993 

0.003 

RSC 

-0.252 

0.074 

-3.406 

0.001 

RSC 

ON 

RE 

0.578 

0.083 

6.963 

0.000 

RD1 

0 . 149 

0.077 

1 . 937 

0.053 

RBV1 

-0.027 

0.079 

-0.341 

0.733 

ICC 

-0.039 

0.079 

-0.494 

0 . 621 

RD1 

ON 

RBV1 

0.317 

0.071 

4.454 

0.000 

ICC 

0.377 

0.065 

5.780 

0.000 

SQ 

ON 

AT  2 

-0.044 

0.053 

-0 . 827 

0.408 

RD2 

0.027 

0.051 

0.527 

0.598 

RBV2 

0.077 

0.052 

1.498 

0.134 

RSC 

ON 

RBV2 

-0.094 

0.059 

-1.599 

0.110 

CBV1 

0.038 

0.071 

0.537 

0.591 

CBV2 

-0.067 

0.073 

-0 . 925 

0.355 

CBV3 

0.051 

0.057 

0 . 901 

0.367 

RD1 

ON 

RD2 

-0 . 110 

0.058 

-1 .884 

0.060 

CBV1 

0.011 

0.075 

0.149 

0 .882 

CBV2 

0.044 

0.074 

0.589 

0.556 

CBV3 

0.032 

0.060 

0.531 

0.596 

RBV1 

WITH 

RE 

0.425 

0.068 

6.281 

0.000 

ICC 

WITH 

RE 

0.576 

0.051 

11.247 

0.000 

RBV1 

0.401 

0.062 

6.444 

0.000 

ATI 

WITH 

RE 

0.262 

0.069 

3.779 

0.000 

RBV1 

0 . 104 

0.071 

1.457 

0 . 145 

ICC 

0.258 

0.065 

3.996 

0.000 

AT  2 

WITH 

RE 

0.234 

0.068 

3.461 

0.001 

RBV1 

0 . 149 

0.067 

2.218 

0.027 

ICC 

0.198 

0.063 

3.130 

0.002 

ATI 

0.253 

0.062 

4.083 

0.000 

RD2 

WITH 

RE 

0.065 

0.069 

0 . 934 

0.350 

RBV1 

-0.010 

0.068 

-0.146 

0 .884 

ICC 

0.092 

0.064 

1.439 

0.150 

ATI 

0.056 

0.064 

0 . 872 

0.383 
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RBV2  WITH 


RE 

-0.055 

0.070 

-0.778 

0.436 

RBV1 

0.005 

0.067 

0.071 

0 . 943 

ICC 

-0.050 

0.065 

-0.778 

0.436 

ATI 

-0.048 

0.064 

-0.747 

0.455 

CBV1  WITH 

RE 

0.039 

0.068 

0.580 

0.562 

RBV1 

-0.198 

0.066 

-2 . 994 

0.003 

ICC 

-0.004 

0.064 

-0.062 

0 . 951 

ATI 

0.003 

0.064 

0.045 

0 . 964 

CBV2  WITH 

RE 

0.022 

0.068 

0.324 

0.746 

RBV1 

-0.207 

0.066 

-3.130 

0.002 

ICC 

-0.099 

0.064 

-1.537 

0.124 

ATI 

0.067 

0.064 

1.052 

0.293 

CBV3  WITH 

RE 

-0.037 

0.068 

-0.549 

0.583 

RBV1 

-0 . 108 

0.067 

-1 . 612 

0.107 

ICC 

0.016 

0.064 

0.253 

0 . 800 

ATI 

0.059 

0.064 

0 . 929 

0.353 

Intercepts 

REA 

4 . 876 

0.231 

21.096 

0.000 

REB 

4.389 

0.210 

20 . 916 

0.000 

RED 

5.058 

0.239 

21.119 

0.000 

ATI  A 

3.549 

0.173 

20.476 

0.000 

AT  1C 

3.497 

0.171 

20.462 

0.000 

AT  ID 

3.693 

0.179 

20 . 601 

0.000 

RBV1C 

2.316 

0.123 

18.792 

0.000 

RBV1D 

2.249 

0.120 

18.735 

0.000 

RBV1E 

2 . 643 

0.136 

19.480 

0.000 

ICCA 

3.570 

0.174 

20.488 

0.000 

ICCB 

3.963 

0.191 

20.751 

0.000 

ICCD 

4.272 

0.205 

20 . 859 

0.000 

SQA 

4.062 

0.266 

15.284 

0.000 

SQC 

5.019 

0.315 

15.953 

0.000 

SQD 

4 . 931 

0.309 

15.956 

0.000 

SQF 

4 . 853 

0.280 

17.332 

0.000 

SQI 

4.303 

0.273 

15.750 

0.000 

SQJ 

4.776 

0.303 

15.746 

0.000 

SQK 

4 . 971 

0.311 

15.986 

0.000 

SQM 

4 . 671 

0.300 

15.572 

0.000 

RSCA 

5.472 

0.294 

18 . 628 

0.000 

RSCC 

5.384 

0.278 

19.351 

0.000 

RSCE 

2.779 

0.152 

18.299 

0.000 

RD1A 

3.590 

0.234 

15.349 

0.000 

RD1B 

3.634 

0.241 

15.098 

0.000 

RD1E 

3.105 

0.202 

15.385 

0.000 

Variances 

RE 

1.000 

0.000 

999.000 

999.000 

RBV1 

1.000 

0.000 

999.000 

999.000 
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ICC 

1.000 

0.000 

999.000 

999.000 

ATI 

1.000 

0.000 

999.000 

999.000 

;idual  Variances 

REA 

0.365 

0.049 

7.450 

0.000 

REB 

0.355 

0.049 

7.277 

0.000 

RED 

0.707 

0.055 

12 . 877 

0.000 

AT1A 

0.237 

0.031 

7 . 608 

0.000 

AT  1C 

0 . 118 

0.024 

4 . 990 

0.000 

AT  ID 

0 . 130 

0.024 

5.364 

0.000 

RBV1C 

0 . 613 

0.057 

10.757 

0.000 

RBV1D 

0.289 

0.051 

5.620 

0.000 

RBV1E 

0.280 

0.051 

5.446 

0.000 

ICCA 

0 .165 

0.029 

5.735 

0.000 

ICCB 

0 . 112 

0.027 

4.221 

0.000 

ICCD 

0.429 

0.046 

9.385 

0.000 

SQA 

0.346 

0.038 

9.193 

0.000 

SQC 

0 . 137 

0.019 

7 . 179 

0.000 

SQD 

0 . 170 

0.022 

7 . 607 

0.000 

SQF 

0.477 

0.046 

10.438 

0.000 

SQI 

0.348 

0.038 

9.196 

0.000 

SQJ 

0 . 179 

0.023 

7.742 

0.000 

SQK 

0 .162 

0.022 

7.512 

0.000 

SQM 

0 . 179 

0.023 

7.708 

0.000 

RSCA 

0.067 

0.059 

1.142 

0.253 

RSCC 

0.347 

0.054 

6.417 

0.000 

RSCE 

0 . 855 

0.045 

19.140 

0.000 

RD1A 

0.242 

0.040 

6.109 

0.000 

RD1B 

0 .161 

0.037 

4.335 

0.000 

RD1E 

0.472 

0.050 

9.413 

0.000 

SQ 

0.345 

0.058 

5.898 

0.000 

RSC 

0 . 612 

0.061 

9.969 

0.000 

RD1 

0 . 663 

0.057 

11.592 

0.000 
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STDY  Standardization 


Two-Tailed 

P-Value 


Estimate  S.E.  Est./S.E. 


RE  BY 


REA 

0.797 

0.031 

25.975 

0.000 

REB 

0 . 803 

0.030 

26.436 

0.000 

RED 

0.542 

0.051 

10 . 689 

0.000 

RBV1 

BY 

RBV1C 

0 . 622 

0.046 

13.601 

0.000 

RBV1D 

0 . 843 

0.031 

27 . 643 

0.000 

RBV1E 

0 . 849 

0.030 

28.030 

0.000 

ICC 

BY 

ICCA 

0 . 914 

0.016 

57 . 963 

0.000 

ICCB 

0 . 942 

0.014 

66.622 

0.000 

ICCD 

0.756 

0.030 

24 . 973 

0.000 

ATI 

BY 

ATI  A 

0 . 873 

0.018 

48 . 857 

0.000 

AT  1C 

0 . 939 

0.013 

74 . 629 

0.000 

AT  ID 

0 . 933 

0.013 

72.045 

0.000 

SQ 

BY 

SQA 

0 . 809 

0.023 

34.756 

0.000 

SQC 

0 . 929 

0.010 

90.246 

0.000 

SQD 

0 . 911 

0.012 

74.245 

0.000 

SQF 

0.724 

0.032 

22 . 933 

0.000 

SQI 

0 . 808 

0.023 

34.526 

0.000 

SQ  J 

0.906 

0.013 

71.239 

0.000 

SQK 

0 . 915 

0.012 

77 . 684 

0.000 

SQM 

0 . 906 

0.013 

70 . 654 

0.000 

RSC 

BY 

RSCA 

0.966 

0.030 

31 . 801 

0.000 

RSCC 

0 . 808 

0.033 

24.168 

0.000 

RSCE 

0.381 

0.059 

6.491 

0.000 

RD1 

BY 

RD1A 

0.871 

0.023 

38.322 

0.000 

RD1B 

0 . 916 

0.020 

45.241 

0.000 

RD1E 

0.727 

0.034 

21.094 

0.000 

SQ 

ON 

RE 

0 . 930 

0.079 

11.773 

0.000 

ATI 

0.048 

0.055 

0 . 887 

0.375 

RD1 

0 .181 

0.066 

2.716 

0.007 

RBV1 

-0.204 

0.068 

-2 . 993 

0.003 

RSC 

-0.252 

0.074 

-3.406 

0.001 

RSC 

ON 

RE 

0.578 

0.083 

6.963 

0.000 

RD1 

0 . 149 

0.077 

1 . 937 

0.053 

RBV1 

-0.027 

0.079 

-0.341 

0.733 

ICC 

-0.039 

0.079 

-0.494 

0 . 621 
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RD1 


ON 


RBV1 

0.317 

0.071 

4.454 

0.000 

ICC 

0.377 

0.065 

5.780 

0.000 

SQ 

ON 

AT  2 

-0.027 

0.033 

-0 . 828 

0.408 

RD2 

0.016 

0.029 

0.527 

0.598 

RBV2 

0.066 

0.044 

1.502 

0.133 

RSC 

ON 

RBV2 

-0.081 

0.050 

-1 . 603 

0.109 

CBV1 

0.004 

0.007 

0.537 

0.591 

CBV2 

-0.064 

0.069 

-0 . 926 

0.354 

CBV3 

0.053 

0.058 

0 . 902 

0.367 

RD1 

ON 

RD2 

-0.063 

0.033 

-1 .891 

0.059 

CBV1 

0.001 

0.008 

0.149 

0 .882 

CBV2 

0.042 

0.070 

0.589 

0.556 

CBV3 

0.032 

0.061 

0.531 

0.595 

RBV1 

WITH 

RE 

0.425 

0.068 

6.281 

0.000 

ICC 

WITH 

RE 

0.576 

0.051 

11.247 

0.000 

RBV1 

0.401 

0.062 

6.444 

0.000 

ATI 

WITH 

RE 

0.262 

0.069 

3.779 

0.000 

RBV1 

0 . 104 

0.071 

1.457 

0 . 145 

ICC 

0.258 

0.065 

3.996 

0.000 

AT  2 

WITH 

RE 

0.234 

0.068 

3.461 

0.001 

RBV1 

0.149 

0.067 

2.218 

0.027 

ICC 

0 .198 

0.063 

3.130 

0.002 

ATI 

0.253 

0.062 

4.083 

0.000 

RD2 

WITH 

RE 

0.065 

0.069 

0 . 934 

0.350 

RBV1 

-0.010 

0.068 

-0 . 146 

0 .884 

ICC 

0.092 

0.064 

1.439 

0.150 

ATI 

0.056 

0.064 

0 . 872 

0.383 

RBV2 

WITH 

RE 

-0.055 

0.070 

-0.778 

0.436 

RBV1 

0.005 

0.067 

0.071 

0 . 943 

ICC 

-0.050 

0.065 

-0.778 

0.436 

ATI 

-0.048 

0.064 

-0.747 

0.455 

CBV1 

WITH 

RE 

0.039 

0.068 

0.580 

0.562 

RBV1 

-0.198 

0.066 

-2 . 994 

0.003 

ICC 

-0.004 

0.064 

-0.062 

0 . 951 

ATI 

0.003 

0.064 

0.045 

0 . 964 
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WITH 


CBV2 


RE 

0.022 

0.068 

0.324 

0.746 

RBV1 

-0.207 

0.066 

-3.130 

0.002 

ICC 

-0.099 

0.064 

-1.537 

0.124 

ATI 

0.067 

0.064 

1.052 

0.293 

CBV3  WITH 

RE 

-0.037 

0.068 

-0.549 

0.583 

RBV1 

-0.108 

0.067 

-1 . 612 

0.107 

ICC 

0.016 

0.064 

0.253 

0 . 800 

ATI 

0.059 

0.064 

0 . 929 

0.353 

Intercepts 

REA 

4 . 876 

0.231 

21.096 

0.000 

REB 

4.389 

0.210 

20 . 916 

0.000 

RED 

5.058 

0.239 

21 . 119 

0.000 

ATI  A 

3.549 

0.173 

20.476 

0.000 

AT  1C 

3.497 

0.171 

20.462 

0.000 

AT  ID 

3.693 

0.179 

20 . 601 

0.000 

RBV1C 

2.316 

0.123 

18.792 

0.000 

RBV1D 

2.249 

0.120 

18.735 

0.000 

RBV1E 

2 . 643 

0.136 

19.480 

0.000 

ICCA 

3.570 

0.174 

20.488 

0.000 

ICCB 

3.963 

0.191 

20.751 

0.000 

ICCD 

4.272 

0.205 

20 . 859 

0.000 

SQA 

4.062 

0.266 

15.284 

0.000 

SQC 

5.019 

0.315 

15.953 

0.000 

SQD 

4 . 931 

0.309 

15.956 

0.000 

SQF 

4 . 853 

0.280 

17.332 

0.000 

SQI 

4.303 

0.273 

15.750 

0.000 

SQJ 

4.776 

0.303 

15.746 

0.000 

SQK 

4 . 971 

0.311 

15.986 

0.000 

SQM 

4 . 671 

0.300 

15.572 

0.000 

RSCA 

5.472 

0.294 

18 . 628 

0.000 

RSCC 

5.384 

0.278 

19.351 

0.000 

RSCE 

2.779 

0.152 

18.299 

0.000 

RD1A 

3.590 

0.234 

15.349 

0.000 

RD1B 

3.634 

0.241 

15.098 

0.000 

RD1E 

3.105 

0.202 

15.385 

0.000 

Variances 

RE 

1.000 

0.000 

999.000 

999.000 

RBV1 

1.000 

0.000 

999.000 

999.000 

ICC 

1.000 

0.000 

999.000 

999.000 

ATI 

1.000 

0.000 

999.000 

999.000 

Residual  Variances 

REA 

0.365 

0.049 

7.450 

0.000 

REB 

0.355 

0.049 

7.277 

0.000 

RED 

0.707 

0.055 

12 . 877 

0.000 

ATI  A 

0.237 

0.031 

7 . 608 

0.000 

AT  1C 

0 . 118 

0.024 

4 . 990 

0.000 

AT  ID 

0 . 130 

0.024 

5.364 

0.000 

RBV1C 

0 . 613 

0.057 

10.757 

0.000 

RBV1D 

0.289 

0.051 

5.620 

0.000 

RBV1E 

0.280 

0.051 

5.446 

0.000 
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ICCA 

0 .165 

0.029 

5.735 

0.000 

ICCB 

0 . 112 

0.027 

4.221 

0.000 

ICCD 

0.429 

0.046 

9.385 

0.000 

SQA 

0.346 

0.038 

9.193 

0.000 

SQC 

0 . 137 

0.019 

7.179 

0.000 

SQD 

0.170 

0.022 

7 . 607 

0.000 

SQF 

0.477 

0.046 

10.438 

0.000 

SQI 

0.348 

0.038 

9.196 

0.000 

SQJ 

0 . 179 

0.023 

7.742 

0.000 

SQK 

0.162 

0.022 

7.512 

0.000 

SQM 

0 . 179 

0.023 

7.708 

0.000 

RSCA 

0.067 

0.059 

1.142 

0.253 

RSCC 

0.347 

0.054 

6.417 

0.000 

RSCE 

0 . 855 

0.045 

19.140 

0.000 

RD1A 

0.242 

0.040 

6.109 

0.000 

RD1B 

0.161 

0.037 

4.335 

0.000 

RD1E 

0.472 

0.050 

9.413 

0.000 

SQ 

0.345 

0.058 

5.898 

0.000 

RSC 

0 . 612 

0.061 

9.969 

0.000 

RD1 

0 . 663 

0.057 

11.592 

0.000 
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STD  Standardization 


Two-Taile< 

Estimate 

S.E. 

Est./S.E. 

P-Value 

RE 

BY 

REA 

0 . 937 

0.067 

14.036 

0.000 

REB 

1.035 

0.073 

14.163 

0.000 

RED 

0 . 630 

0.074 

8.520 

0.000 

RBV1 

BY 

RBV1C 

1 .197 

0.120 

9.984 

0.000 

RBV1D 

1.548 

0.105 

14.772 

0.000 

RBV1E 

1.428 

0.096 

14 .894 

0.000 

ICC 

BY 

ICCA 

1.344 

0.074 

18.082 

0.000 

ICCB 

1.284 

0.068 

19.014 

0.000 

ICCD 

1.003 

0.074 

13.526 

0.000 

ATI 

BY 

ATI  A 

1.302 

0.077 

16.955 

0.000 

AT  1C 

1.435 

0.075 

19.103 

0.000 

AT  ID 

1.375 

0.073 

18.891 

0.000 

SQ 

BY 

SQA 

1.074 

0.070 

15.320 

0.000 

SQC 

1.072 

0.056 

19.146 

0.000 

SQD 

1.058 

0.057 

18.519 

0.000 

SQF 

0 . 840 

0.064 

13.071 

0.000 

SQI 

1.035 

0.068 

15.286 

0.000 

SQ  J 

1 . 101 

0.060 

18.363 

0.000 

SQK 

1.065 

0.057 

18 . 672 

0.000 

SQM 

1 . 112 

0.061 

18.352 

0.000 

RSC 

BY 

RSCA 

1 . 104 

0.063 

17.522 

0.000 

RSCC 

0 . 943 

0.068 

13.969 

0.000 

RSCE 

0 . 663 

0.113 

5.854 

0.000 

RD1 

BY 

RD1A 

1.293 

0.079 

16.285 

0.000 

RD1B 

1.329 

0.076 

17.565 

0.000 

RD1E 

1 . 153 

0.091 

12 . 604 

0.000 

SQ 

ON 

RE 

0 . 930 

0.079 

11.773 

0.000 

ATI 

0.048 

0.055 

0 .887 

0.375 

RD1 

0 .181 

0.066 

2.716 

0.007 

RBV1 

-0.204 

0.068 

-2 . 993 

0.003 

RSC 

-0.252 

0.074 

-3.406 

0.001 

RSC 

ON 

RE 

0.578 

0.083 

6.963 

0.000 

RD1 

0 . 149 

0.077 

1 . 937 

0.053 

RBV1 

-0.027 

0.079 

-0.341 

0.733 

ICC 

-0.039 

0.079 

-0.494 

0 . 621 
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RD1 


ON 


RBV1 

0.317 

0.071 

4.454 

0.000 

ICC 

0.377 

0.065 

5.780 

0.000 

SQ 

ON 

AT  2 

-0.027 

0.033 

-0 . 828 

0.408 

RD2 

0.016 

0.029 

0.527 

0.598 

RBV2 

0.066 

0.044 

1.502 

0.133 

RSC 

ON 

RBV2 

-0.081 

0.050 

-1 . 603 

0.109 

CBV1 

0.004 

0.007 

0.537 

0.591 

CBV2 

-0.064 

0.069 

-0 . 926 

0.354 

CBV3 

0.053 

0.058 

0 . 902 

0.367 

RD1 

ON 

RD2 

-0.063 

0.033 

-1 .891 

0.059 

CBV1 

0.001 

0.008 

0.149 

0 .882 

CBV2 

0.042 

0.070 

0.589 

0.556 

CBV3 

0.032 

0.061 

0.531 

0.595 

RBV1 

WITH 

RE 

0.425 

0.068 

6.281 

0.000 

ICC 

WITH 

RE 

0.576 

0.051 

11.247 

0.000 

RBV1 

0.401 

0.062 

6.444 

0.000 

ATI 

WITH 

RE 

0.262 

0.069 

3.779 

0.000 

RBV1 

0 . 104 

0.071 

1.457 

0 . 145 

ICC 

0.258 

0.065 

3.996 

0.000 

AT  2 

WITH 

RE 

0.381 

0.109 

3.505 

0.000 

RBV1 

0.243 

0.109 

2.229 

0.026 

ICC 

0.322 

0.102 

3.163 

0.002 

ATI 

0.411 

0.099 

4.156 

0.000 

RD2 

WITH 

RE 

0 . 112 

0.120 

0 . 935 

0.350 

RBV1 

-0.017 

0.117 

-0 . 146 

0 .884 

ICC 

0 .160 

0.111 

1.442 

0 . 149 

ATI 

0.096 

0.110 

0 . 872 

0.383 

RBV2 

WITH 

RE 

-0.064 

0.082 

-0.779 

0.436 

RBV1 

0.006 

0.078 

0.071 

0 . 943 

ICC 

-0.059 

0.075 

-0.779 

0.436 

ATI 

-0.055 

0.074 

-0.747 

0.455 

CBV1 

WITH 

RE 

0.388 

0 . 668 

0.580 

0.562 

RBV1 

-1 . 957 

0 . 647 

-3.022 

0.003 

ICC 

-0.039 

0 . 636 

-0.062 

0 . 951 

ATI 

0.028 

0 . 629 

0.045 

0 . 964 
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WITH 


CBV2 


RE 

0.023 

0.072 

0.324 

0.746 

RBV1 

-0.219 

0.069 

-3.163 

0.002 

ICC 

-0.104 

0.068 

-1.540 

0.123 

ATI 

0.071 

0.067 

1.053 

0.292 

CBV3  WITH 

RE 

-0.036 

0.066 

-0.549 

0.583 

RBV1 

-0.106 

0.065 

-1 . 616 

0.106 

ICC 

0.016 

0.063 

0.253 

0 . 800 

ATI 

0.058 

0.062 

0 . 930 

0.353 

Intercepts 

REA 

5.729 

0.074 

77.242 

0.000 

REB 

5.654 

0.081 

69.559 

0.000 

RED 

5.883 

0.074 

79.160 

0.000 

ATI  A 

5.292 

0.093 

56.793 

0.000 

AT  1C 

5.342 

0.095 

56.259 

0.000 

AT  ID 

5.442 

0.092 

59.376 

0.000 

RBV1C 

4.454 

0.122 

36.496 

0.000 

RBV1D 

4 . 129 

0.115 

35.957 

0.000 

RBV1E 

4.446 

0.105 

42.267 

0.000 

ICCA 

5.250 

0.092 

57.090 

0.000 

ICCB 

5.400 

0.085 

63.504 

0.000 

ICCD 

5.671 

0.084 

67 . 610 

0.000 

SQA 

5.396 

0.257 

21.033 

0.000 

SQC 

5.791 

0.252 

22 . 937 

0.000 

SQD 

5.724 

0.250 

22 . 939 

0.000 

SQF 

5.633 

0.203 

27.702 

0.000 

SQI 

5.512 

0.247 

22.303 

0.000 

SQJ 

5.800 

0.260 

22.320 

0.000 

SQK 

5.783 

0.251 

23.024 

0.000 

SQM 

5.734 

0.263 

21 . 835 

0.000 

RSCA 

6.255 

0.183 

34 . 116 

0.000 

RSCC 

6.284 

0.162 

38 . 831 

0.000 

RSCE 

4 . 838 

0.151 

32.044 

0.000 

RD1A 

5.333 

0.249 

21.392 

0.000 

RD1B 

5.271 

0.254 

20.727 

0.000 

RD1E 

4 . 926 

0.229 

21.484 

0.000 

Variances 

RE 

1.000 

0.000 

999.000 

999.000 

RBV1 

1.000 

0.000 

999.000 

999.000 

ICC 

1.000 

0.000 

999.000 

999.000 

ATI 

1.000 

0.000 

999.000 

999.000 

Residual  Variances 

REA 

0.503 

0.063 

7 . 962 

0.000 

REB 

0.589 

0.076 

7.789 

0.000 

RED 

0 . 956 

0.093 

10.271 

0.000 

ATI  A 

0.528 

0.060 

8.793 

0.000 

AT  1C 

0.275 

0.051 

5.436 

0.000 

AT  ID 

0.281 

0.048 

5.911 

0.000 

RBV1C 

2.266 

0.232 

9.746 

0.000 

RBV1D 

0 . 974 

0.164 

5.958 

0.000 

RBV1E 

0.792 

0.137 

5.766 

0.000 
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ICCA 

0.357 

0.056 

6.344 

0.000 

ICCB 

0.209 

0.047 

4.485 

0.000 

ICCD 

0.756 

0.076 

9.908 

0.000 

SQA 

0 . 610 

0.059 

10.275 

0.000 

SQC 

0 . 183 

0.021 

8 . 605 

0.000 

SQD 

0.229 

0.025 

9.138 

0.000 

SQF 

0 . 642 

0.061 

10.563 

0.000 

SQI 

0.570 

0.056 

10.271 

0.000 

SQJ 

0.263 

0.028 

9.318 

0.000 

SQK 

0.219 

0.024 

9.023 

0.000 

SQM 

0.270 

0.029 

9.240 

0.000 

RSCA 

0.088 

0.076 

1.145 

0.252 

RSCC 

0.472 

0.070 

6.723 

0.000 

RSCE 

2.592 

0.242 

10.709 

0.000 

RD1A 

0.533 

0.080 

6.668 

0.000 

RD1B 

0.338 

0.074 

4.585 

0.000 

RD1E 

1.187 

0.123 

9.617 

0.000 

SQ 

0.345 

0.058 

5.898 

0.000 

RSC 

0 . 612 

0.061 

9.969 

0.000 

RD1 

0 . 663 

0.057 

11.592 

0.000 

R- SQUARE 

Observed 

Two-Taile< 

Variable 

Estimate 

S.E. 

Est./S.E. 

P-Value 

REA 

0 . 635 

0.049 

12 . 987 

0.000 

REB 

0 . 645 

0.049 

13.218 

0.000 

RED 

0.293 

0.055 

5.344 

0.000 

ATI  A 

0.763 

0.031 

24.429 

0.000 

AT  1C 

0 .882 

0.024 

37.314 

0.000 

AT  ID 

0.870 

0.024 

36.022 

0.000 

RBV1C 

0.387 

0.057 

6.800 

0.000 

RBV1D 

0.711 

0.051 

13.821 

0.000 

RBV1E 

0.720 

0.051 

14.015 

0.000 

ICCA 

0 . 835 

0.029 

28 . 981 

0.000 

ICCB 

0 .888 

0.027 

33.311 

0.000 

ICCD 

0.571 

0.046 

12.486 

0.000 

SQA 

0 . 654 

0.038 

17.378 

0.000 

SQC 

0 .863 

0.019 

45.123 

0.000 

SQD 

0.830 

0.022 

37.123 

0.000 

SQF 

0.523 

0.046 

11.466 

0.000 

SQI 

0 . 652 

0.038 

17.263 

0.000 

SQJ 

0 . 821 

0.023 

35.619 

0.000 

SQK 

0.838 

0.022 

38 . 842 

0.000 

SQM 

0 . 821 

0.023 

35.327 

0.000 

RSCA 

0 . 933 

0.059 

15.900 

0.000 

RSCC 

0 . 653 

0.054 

12.084 

0.000 

RSCE 

0 . 145 

0.045 

3.245 

0.001 

RD1A 

0.758 

0.040 

19.161 

0.000 

RD1B 

0.839 

0.037 

22 . 621 

0.000 

RD1E 

0.528 

0.050 

10.547 

0.000 

Latent 

Two-Taile< 

Variable 

Estimate 

S.E. 

Est./S.E. 

P-Value 
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SQ 

0 . 655 

0.058 

11.218 

0.000 

RSC 

0.388 

0.061 

6.322 

0.000 

RD1 

0.337 

0.057 

5.883 

0.000 

QUALITY  OF  NUMERICAL  RESULTS 

Condition  Number  for  the  Information  Matrix  0.136E-04 

(ratio  of  smallest  to  largest  eigenvalue) 


TOTAL,  TOTAL  INDIRECT,  SPECIFIC  INDIRECT,  AND  DIRECT  EFFECTS 


Estimate 

S.E. 

Est./S.E. 

Two-Taile< 

P-Value 

Effects  from  RE  to 

SQ 

Total 

Total  indirect 

0.899 
-0 .167 

0.103 

0.064 

8.735 

-2.592 

0.000 

0.010 

Specific  indirect 

SQ 

RSC 

RE 

-0.167 

0.064 

-2.592 

0.010 

Direct 

SQ 

RE 

1.066 

0.129 

8.256 

0.000 

Effects  from  RD1  to 

SQ 

Total 

Total  indirect 

0 . 119 
-0.031 

0.054 

0.018 

2 .194 
-1.732 

0.028 

0.083 

Specific  indirect 

SQ 

RSC 

RD1 

-0.031 

0.018 

-1.732 

0.083 

Direct 

SQ 

RD1 

0.150 

0.055 

2.728 

0.006 

Effects  from  RD2  to 

SQ 

Total 

Total  indirect 

0.007 

-0.010 

0.032 

0.007 

0.220 

-1.433 

0 . 826 
0.152 

Specific  indirect 
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SQ 

RD1 

RD2 


-0.012  0.008  -1.555  0.120 


SQ 

RSC 

RD1 

RD2 


Direct 

SQ 

RD2 


0.003  0.002  1.277  0.202 


0.017  0.032  0.527  0.598 


Effects  from  RBV1  to 
Total 

Total  indirect 

Specific  indirect 

SQ 

RSC 

RBV1 

SQ 

RD1 

RBV1 

SQ 

RSC 

RD1 

RBV1 

Direct 

SQ 

RBV1 


SQ 

-0.136  0.058 

0.047  0.028 

0.006  0.018 

0.051  0.023 

-0.011  0.007 

-0.183  0.063 


-2.355  0.019 

1.676  0.094 


0.335  0.738 


2.223  0.026 


-1.592  0.111 


-2.928  0.003 


Effects  from  RBV2  to  SQ 

Total  0.093  0.046  2.022 

Total  indirect  0.022  0.015  1.445 


0.043 

0.148 


Specific  indirect 

SQ 

RSC 

RBV2  0.022  0.015  1.445  0.148 

Direct 

SQ 

RBV2  0.071  0.048  1.497  0.134 


Effects  from  RBV1  to  RSC 
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Total 

Total  indirect 


Specific  indirect 
RSC 
RD1 
RBV1 

Direct 

RSC 

RBV1 


Effects  from  CBV1  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV1 

Direct 

RSC 

CBV1 


Effects  from  CBV2  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV2 

Direct 

RSC 

CBV2 

Effects  from  CBV3  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV3 


0.019  0.070 

0.044  0.025 

0.044  0.025 

-0.025  0.073 

RSC 

0.004  0.008 

0.000  0.001 

0.000  0.001 

0.004  0.008 

RSC 

-0.063  0.077 

0.007  0.012 

0.007  0.012 

-0.070  0.076 

RSC 

0.064  0.065 

0.005  0.010 

0.005  0.010 


0.268  0.789 

1.753  0.080 


1.753  0.080 


-0.341  0.733 


0.555  0.579 

0.148  0.882 


0.148  0.882 


0.538  0.591 


-0.824  0.410 

0.564  0.573 


0.564  0.573 


-0.922  0.356 


0.977  0.328 

0.513  0.608 


0.513  0.608 
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Direct 

RSC 

CBV3 


0.058 


0.064 


0 . 902 


0.367 


Effects  from  ICC  to  RSC 

Total  0.014  0.065 

Total  indirect  0.046  0.025 


0.217  0.828 

1.836  0.066 


Specific  indirect 

RSC 

RD1 

ICC  0.046  0.025  1.836  0.066 


Direct 

RSC 

ICC  -0.032  0.065  -0.494  0.621 


STANDARDIZED  TOTAL,  TOTAL  INDIRECT,  SPECIFIC  INDIRECT,  AND  DIRECT 
EFFECTS 


STDYX  Standardization 


Estimate 

Effects  from  RE  to  SQ 


Total  0.784 

Total  indirect  -0.145 

Specific  indirect 

SQ 

RSC 

RE  -0.145 

Direct 

SQ 

RE  0.930 

Effects  from  RD1  to  SQ 

Total  0.143 

Total  indirect  -0.038 


Two-Tailed 

S.E.  Est./S.E.  P-Value 


0.060  12.978  0.000 

0.054  -2.674  0.007 


0.054  -2.674  0.007 


0.079  11.773  0.000 


0.066  2.181  0.029 

0.022  -1.741  0.082 


Specific  indirect 

SQ 

RSC 
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RD1 


-0.038 


0.022 


1.741 


0.082 


Direct 

SQ 

RD1 


Effects  from  RD2  to 
Total 

Total  indirect 

Specific  indirect 

SQ 

RD1 

RD2 

SQ 

RSC 

RD1 

RD2 

Direct 

SQ 

RD2 


0.181  0.066 


0.011  0.051 

-0.016  0.011 


-0.020  0.013 


0.004  0.003 


0.027  0.051 


2.716  0.007 


0.220  0.826 

-1.428  0.153 


-1.550  0.121 


1.278  0.201 


0.527  0.598 


Effects  from  RBV1  to 
Total 

Total  indirect 

Specific  indirect 

SQ 
RSC 
RBV1 

SQ 
RD1 
RBV1 

SQ 
RSC 
RD1 
RBV1 

Direct 
SQ 

RBV1 


SQ 

-0.152  0.064 

0.052  0.031 

0.007  0.020 

0.057  0.025 

-0.012  0.007 

-0.204  0.068 


-2.394  0.017 

1.682  0.093 


0.335  0.738 


2.249  0.025 


-1.611  0.107 


-2.993  0.003 


Effects  from  RBV2  to  SQ 


Total 


0.101 


0.050  2.027  0.043 
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Total  indirect 


0.024 


0.016 


1.449 


0.147 


Specific  indirect 


SQ 

RSC 

RBV2 

Direct 

SQ 

RBV2 


Effects  from  RBV1  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

RBV1 

Direct 

RSC 

RBV1 


Effects  from  CBV1  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV1 

Direct 

RSC 

CBV1 


Effects  from  CBV2  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV2 


0.024  0.016 

0.077  0.052 

RSC 

0.020  0.076 

0.047  0.027 

0.047  0.027 

-0.027  0.079 

RSC 

0.040  0.072 

0.002  0.011 

0.002  0.011 

0.038  0.071 

RSC 

-0.061  0.073 

0.007  0.012 

0.007  0.012 


1.449  0.147 


1.498  0.134 


0.268  0.789 

1.766  0.077 


1.766  0.077 


-0.341  0.733 


0.555  0.579 

0.148  0.882 


0.148  0.882 


0.537  0.591 


-0.827  0.408 

0.563  0.573 


0.563  0.573 
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Direct 

RSC 

CBV2 

-0.067 

0.073 

-0 . 925 

0.355 

Effects  from  CBV3  to 

RSC 

Total 

Total  indirect 
Specific  indirect 

0.056 

0.005 

0.058 

0.009 

0 . 976 
0.512 

0.329 
0 . 608 

RSC 

RD1 

CBV3 

0.005 

0.009 

0.512 

0 . 608 

Direct 

RSC 

CBV3 

0.051 

0.057 

0 . 901 

0.367 

Effects  from  ICC  to 

RSC 

Total 

Total  indirect 

0.017 

0.056 

0.079 

0.031 

0.217 

1 . 830 

0 . 828 
0.067 

Specific  indirect 

RSC 

RD1 

ICC 

0.056 

0.031 

1 . 830 

0.067 

Direct 

RSC 

ICC 

-0.039 

0.079 

-0.494 

0 . 621 

STDY  Standardization 

Estimate 

S.E. 

Est./S.E. 

Two-Taile< 

P-Value 

Effects  from  RE  to  SQ 

Total 

Total  indirect 

0.784 
-0 . 145 

0.060 

0.054 

12 . 978 
-2 . 674 

0.000 

0.007 

Specific  indirect 

SQ 

RSC 

RE 

-0 . 145 

0.054 

-2 . 674 

0.007 

Direct 

SQ 
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RE 


0 . 930 


0.079 


11.773 


0.000 


Effects  from  RD1  to  SQ 

Total  0.143  0.066  2.181  0.029 

Total  indirect  -0.038  0.022  -1.741  0.082 


Specific  indirect 

SQ 

RSC 

RD1  -0.038  0.022  -1.741  0.082 

Direct 

SQ 

RD1  0.181  0.066  2.716  0.007 


Effects  from  RD2  to  SQ 

Total  0.006 

Total  indirect  -0.009 

Specific  indirect 

SQ 

RD1 

RD2  -0.011 

SQ 

RSC 

RD1 

RD2  0.002 

Direct 

SQ 

RD2  0.016 


0.029  0.220 

0.006  -1.404 


0.008  -1.519 


0.002  1.261 


0.029  0.526 


0 . 826 
0.160 


0.129 


0.207 


0.599 


Effects  from  RBV1  to 
Total 

Total  indirect 

Specific  indirect 

SQ 

RSC 

RBV1 

SQ 

RD1 

RBV1 


SQ 

-0.152  0.064 

0.052  0.031 

0.007  0.020 

0.057  0.025 


-2.394  0.017 

1.682  0.093 


0.335  0.738 


2.249  0.025 
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SQ 

RSC 

RD1 

RBV1 

Direct 

SQ 

RBV1 


Effects  from  RBV2  to 
Total 

Total  indirect 

Specific  indirect 

SQ 

RSC 

RBV2 

Direct 

SQ 

RBV2 


Effects  from  RBV1  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

RBV1 

Direct 

RSC 

RBV1 


Effects  from  CBV1  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV1 

Direct 

RSC 


-0.012  0.007 

-0.204  0.068 

SQ 

0.087  0.043 

0.020  0.014 

0.020  0.014 


0.066  0.044 


RSC 

0.020  0.076 

0.047  0.027 

0.047  0.027 

-0.027  0.079 

RSC 

0.004  0.019 

0.000  0.001 

0.000  0.001 


-1.611  0.107 


-2.993  0.003 


2.020  0.043 

1.446  0.148 


1.446  0.148 


1.496  0.135 


0.268  0.789 

1.766  0.077 


1.766  0.077 


-0.341  0.733 


0.209  0.835 

0.124  0.901 


0.124  0.901 
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CBV1 


0.004 


0.019 


0.208 


0 . 835 


Effects  from  CBV2  to  RSC 


Total 

Total  indirect 


0.057  0.069 

0.006  0.011 


-0.826  0.409 

0.563  0.573 


Specific  indirect 

RSC 

RD1 

CBV2  0.006  0.011  0.563  0.573 

Direct 

RSC 

CBV2  -0.064  0.069  -0.925  0.355 


Effects  from  CBV3  to  RSC 


Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV3 

Direct 

RSC 

CBV3 


Effects  from  ICC  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

ICC 

Direct 

RSC 

ICC 


0.058  0.059 

0.005  0.009 


0.005  0.009 


0.053  0.058 


RSC 

0.017  0.079 

0.056  0.031 


0.056  0.031 


-0.039  0.079 


0.976  0.329 

0.512  0.608 


0.512  0.608 


0.901  0.367 


0.217  0.828 

1.830  0.067 


1.830  0.067 


-0.494  0.621 


STD  Standardization 
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Estimate 


S.E. 


Est./S.E. 


Two-Tailed 

P-Value 


Effects  from  RE  to  SQ 

Total  0.784  0.060  12.978  0.000 

Total  indirect  -0.145  0.054  -2.674  0.007 


Specific  indirect 

SQ 

RSC 

RE  -0.145  0.054  -2.674  0.007 

Direct 

SQ 

RE  0.930  0.079  11.773  0.000 


Effects  from  RD1  to  SQ 

Total  0.143  0.066  2.181  0.029 

Total  indirect  -0.038  0.022  -1.741  0.082 

Specific  indirect 

SQ 

RSC 

RD1  -0.038  0.022  -1.741  0.082 


Direct 

SQ 

RD1  0.181  0.066  2.716  0.007 


Effects  from  RD2  to  SQ 


Total  0.006  0.029  0.220  0.826 

Total  indirect  -0.009  0.006  -1.431  0.153 

Specific  indirect 

SQ 

RD1 

RD2  -0.011  0.007  -1.553  0.120 

SQ 

RSC 

RD1 
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RD2 


0.002 


0.002 


1.280 


0.200 


Direct 

SQ 

RD2  0.016  0.029  0.527  0.598 


Effects  from  RBV1 
Total 

Total  indirect 


SQ 

-0.152  0.064 

0.052  0.031 


-2.394  0.017 

1.682  0.093 


Specific  indirect 


SQ 

RSC 

RBV1 

SQ 

RD1 

RBV1 

SQ 

RSC 

RD1 

RBV1 

Direct 

SQ 

RBV1 


Effects  from  RBV2  to 
Total 

Total  indirect 

Specific  indirect 

SQ 

RSC 

RBV2 

Direct 

SQ 

RBV2 


Effects  from  RBV1  to 
Total 

Total  indirect 


0.007  0.020 


0.057  0.025 

-0.012  0.007 

-0.204  0.068 

SQ 

0.087  0.043 

0.020  0.014 

0.020  0.014 


0.066  0.044 


RSC 

0.020  0.076 

0.047  0.027 


0.335  0.738 


2.249  0.025 


-1.611  0.107 


-2.993  0.003 


2.036  0.042 

1.452  0.146 


1.452  0.146 


1.502  0.133 


0.268  0.789 

1.766  0.077 


Specific  indirect 
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RSC 

RD1 

RBV1 

Direct 

RSC 

RBV1 


Effects  from  CBV1  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV1 

Direct 

RSC 

CBV1 


Effects  from  CBV2  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV2 

Direct 

RSC 

CBV2 


Effects  from  CBV3  to 
Total 

Total  indirect 

Specific  indirect 

RSC 

RD1 

CBV3 

Direct 

RSC 


0.047  0.027 

-0.027  0.079 

RSC 

0.004  0.007 

0.000  0.001 

0.000  0.001 

0.004  0.007 

RSC 

-0.057  0.069 

0.006  0.011 

0.006  0.011 

-0.064  0.069 

RSC 

0.058  0.059 

0.005  0.009 

0.005  0.009 


1.766  0.077 


-0.341  0.733 


0.555  0.579 

0.148  0.882 


0.148  0.882 


0.537  0.591 


-0.827  0.408 

0.563  0.573 


0.563  0.573 


-0.926  0.354 


0.977  0.328 

0.513  0.608 


0.513  0.608 
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CBV3 


0.053 


0.058 


0.902 


0.367 


Effects  from  ICC  to  RSC 


Total  0.017 

Total  indirect  0.056 

Specific  indirect 

RSC 

RD1 

ICC  0.056 


Direct 

RSC 

ICC  -0.039 


0.079  0.217 

0.031  1.830 


0.031  1.830 


0.079  -0.494 


Beginning  Time: 
Ending  Time: 
Elapsed  Time: 


17:05:40 

17:05:42 

00:00:02 


MUTHEN  &  MUTHEN 
3463  Stoner  Ave . 

Los  Angeles,  CA  90066 

Tel:  (310)  391-9971 
Fax:  (310)  391-8971 
Web:  www.StatModel.com 
Support:  Support@StatModel.com 

Copyright  (c)  1998-2010  Muthen  &  Muthen 
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APPENDIX  O.  HYPOTHESIZED  MODEL  MODIFICATION 

INDICES 


Minimum  M.I.  value  for  printing  the  modification  index  15.000 


M.I. 

E.P.C. 

Std  E.P.C. 

StdYX  E.P.C 

ON /BY 

Statements 

RE 

ON  RD1 

/ 

RD1 

BY  RE 

22.174 

0.273 

0.377 

0.377 

RBV1 

ON  RD1 

/ 

RD1 

BY  RBV1 

20 . 913 

-1.278 

-1.382 

-1.382 

ICC 

ON  RD1 

/ 

RD1 

BY  ICC 

18.222 

-0 . 685 

-0 . 659 

-0 . 659 

RD1 

ON  RE 

/ 

RE 

BY  RD1 

22.371 

0.578 

0.418 

0.418 

RD1 

ON  SQ 

/ 

SQ 

BY  RD1 

20.238 

0.559 

0.464 

0.464 

RD1 

ON  RSC 

/ 

RSC 

BY  RD1 

19.474 

0.730 

0 . 623 

0 . 623 

ON  Statements 

RE 

ON  RD1A 

19.308 

0.191 

0.204 

0.303 

RE 

ON  RD1B 

18.729 

0.198 

0.212 

0.307 

RBV1 

ON  RD1B 

16.196 

-0 . 610 

-0.510 

-0.739 

ICC 

ON  RD1A 

16.416 

-0.387 

-0.288 

-0.428 

RD1 

ON  REA 

20.017 

0.331 

0.256 

0.301 

RD1 

ON  SQM 

18.276 

0.405 

0.313 

0.384 

RD1 

ON  RSCA 

19.559 

0.706 

0.546 

0 . 624 

SQC 

ON  SQK 

27.761 

0.414 

0.414 

0.417 

SQD 

ON  SQM 

18.361 

0.317 

0.317 

0.335 

SQK 

ON  SQC 

27.761 

0.497 

0.497 

0.493 

SQM 

ON  SQD 

18.360 

0.374 

0.374 

0.353 

WITH 

Statements 

RD1 

WITH  RE 

22.174 

0.303 

0.307 

0.307 

RD1 

WITH  RBV1 

20 . 913 

-1.419 

-1 . 125 

-1.125 

RD1 

WITH  ICC 

18.222 

-0.760 

-0.537 

-0.537 

SQK 

WITH  SQC 

27.761 

0.091 

0.091 

0.453 

SQM 

WITH  SQD 

18.360 

0.086 

0.086 

0.344 
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APPENDIX  Q.  TRIMMED  MODEL  MODIFICATION  INDICES 


Minimum  M.I.  value 

for 

printing  the 

modification 

index 

15.000 

M.I . 

E.P.C.  Std 

E.P.C. 

StdYX  E.P.C 

ON/BY 

Statements 

RE 

ON  RD1 

/ 

RD1 

BY  RE 

23.206 

0.282 

0.389 

0.389 

RBV1 

ON  RD1 

/ 

RD1 

BY  RBV1 

22 . 807 

-1.548 

-1 . 679 

-1 . 679 

ICC 

ON  RD1 

/ 

RD1 

BY  ICC 

23.047 

-0 . 834 

-0 . 805 

-0 . 805 

RD1 

ON  RE 

/ 

RE 

BY  RD1 

23.211 

0.576 

0.417 

0.417 

RD1 

ON  SQ 

/ 

SQ 

BY  RD1 

20.179 

0.533 

0.443 

0.443 

ON  Statements 

RE 

ON  RD1A 

20.798 

0.202 

0.215 

0.320 

RE 

ON  RD1B 

19.480 

0.205 

0.219 

0.318 

ICC 

ON  RD1A 

19.886 

-0.448 

-0.334 

-0.497 

RD1 

ON  REA 

20.382 

0.332 

0.257 

0.302 

RD1 

ON  SQD 

15.180 

0.381 

0.295 

0.342 

RD1 

ON  SQM 

18.535 

0.396 

0.306 

0.375 

SQC 

ON  SQK 

27.743 

0.414 

0.414 

0.417 

SQD 

ON  SQM 

18.317 

0.317 

0.317 

0.336 

SQK 

ON  SQC 

27.742 

0.497 

0.497 

0.493 

SQM 

ON  SQD 

18.317 

0.373 

0.373 

0.353 

WITH 

Statements 

RD1 

WITH  RE 

23.205 

0.316 

0.319 

0.319 

RD1 

WITH  RBV1 

22 . 805 

-1.732 

-1.374 

-1.374 

RD1 

WITH  ICC 

23.047 

-0 . 934 

-0 . 659 

-0 . 659 

SQK 

WITH  SQC 

27.743 

0.091 

0.091 

0.453 

SQM 

WITH  SQD 

18.317 

0.086 

0.086 

0.344 
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2003  -  2010  SPONSORED  RESEARCH  TOPICS 


Acquisition  Management 

■  Acquiring  Combat  Capability  via  Public-Private  Partnerships  (PPPs) 

■  BCA:  Contractor  vs.  Organic  Growth 

■  Defense  Industry  Consolidation 

■  EU-US  Defense  Industrial  Relationships 

■  Knowledge  Value  Added  (KVA)  +  Real  Options  (RO)  Applied  to 
Shipyard  Planning  Processes 

■  Managing  the  Services  Supply  Chain 

■  MOSA  Contracting  Implications 

■  Portfolio  Optimization  via  KVA  +  RO 

■  Private  Military  Sector 

■  Software  Requirements  for  OA 

■  Spiral  Development 

■  Strategy  for  Defense  Acquisition  Research 

■  The  Software,  Hardware  Asset  Reuse  Enterprise  (SHARE)  repository 

Contract  Management 

■  Commodity  Sourcing  Strategies 

■  Contracting  Government  Procurement  Functions 

■  Contractors  in  21  st-century  Combat  Zone 

■  Joint  Contingency  Contracting 

■  Model  for  Optimizing  Contingency  Contracting,  Planning  and  Execution 

■  Navy  Contract  Writing  Guide 

■  Past  Performance  in  Source  Selection 

■  Strategic  Contingency  Contracting 

■  Transforming  DoD  Contract  Closeout 

■  USAF  Energy  Savings  Performance  Contracts 

■  USAF  IT  Commodity  Council 

■  USMC  Contingency  Contracting 
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Financial  Management 

■  Acquisitions  via  Leasing:  MPS  case 

■  Budget  Scoring 

■  Budgeting  for  Capabilities-based  Planning 

■  Capital  Budgeting  for  the  DoD 

■  Energy  Saving  Contracts/DoD  Mobile  Assets 

■  Financing  DoD  Budget  via  PPPs 

■  Lessons  from  Private  Sector  Capital  Budgeting  for  DoD  Acquisition 
Budgeting  Reform 

■  PPPs  and  Government  Financing 

■  ROI  of  Information  Warfare  Systems 

■  Special  Termination  Liability  in  MDAPs 

■  Strategic  Sourcing 

■  Transaction  Cost  Economics  (TCE)  to  Improve  Cost  Estimates 

Human  Resources 

■  Indefinite  Reenlistment 

■  Individual  Augmentation 

■  Learning  Management  Systems 

■  Moral  Conduct  Waivers  and  First-tern  Attrition 

■  Retention 

■  The  Navy’s  Selective  Reenlistment  Bonus  (SRB)  Management  System 

■  Tuition  Assistance 

Logistics  Management 

■  Analysis  of  LAV  Depot  Maintenance 

■  Army  LOG  MOD 

■  ASDS  Product  Support  Analysis 

■  Cold-chain  Logistics 

■  Contractors  Supporting  Military  Operations 

■  Diffusion/Variability  on  Vendor  Performance  Evaluation 

■  Evolutionary  Acquisition 

■  Lean  Six  Sigma  to  Reduce  Costs  and  Improve  Readiness 
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■  Naval  Aviation  Maintenance  and  Process  Improvement  (2) 

■  Optimizing  CIWS  Lifecycle  Support  (LCS) 

■  Outsourcing  the  Pearl  Harbor  MK-48  Intermediate  Maintenance 
Activity 

■  Pallet  Management  System 

-  PBL  (4) 

■  Privatization-NOSL/NAWCI 

-  RFID  (6) 

■  Risk  Analysis  for  Performance-based  Logistics 

■  R-TOC  AEGIS  Microwave  Power  Tubes 

■  Sense-and-Respond  Logistics  Network 

■  Strategic  Sourcing 

Program  Management 

■  Building  Collaborative  Capacity 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module 
Acquisition 

■  Collaborative  IT  Tools  Leveraging  Competence 

■  Contractor  vs.  Organic  Support 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

■  KVA  Applied  to  AEGIS  and  SSDS 

■  Managing  the  Service  Supply  Chain 

■  Measuring  Uncertainty  in  Earned  Value 

■  Organizational  Modeling  and  Simulation 

■  Public-Private  Partnership 

■  Terminating  Your  Own  Program 

■  Utilizing  Collaborative  and  Three-dimensional  Imaging  Technology 

A  complete  listing  and  electronic  copies  of  published  research  are  available  on  our 
website:  www.acquisitionresearch.org 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 

555  DYER  ROAD,  INGERSOLL  HALL 

MONTEREY,  CALIFORNIA  93943 


www.acquisitionresearch.org 


